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FOREWORD 


The  decline  in  performance  caused  by  forgetting  tasks  is  a 
critical  training  problem  in  the  Army.  One  of  the  trainer's 
primary  responsibilities  is  to  ensure  that  his  or  her  soldiers 
remain  proficient  on  tasks  they  have  already  learned.  This 
means  conducting  periodic  refresher  training  since  soldiers  for¬ 
get  tasks  not  practiced  in  the  unit  on  a  regular  basis.  Unfor¬ 
tunately,  unit  training  resources  are  scarce,  and  no  method  has 
been  available  for  helping  the  trainer  identify  tasks  that  either 
have  been,  or  are  about  to  be,  forgotten.  Without  such  informa¬ 
tion  it  is  difficult  to  target  refresher  training  effectively, 
and  thereby  obtain  maximum  payoff  from  the  limited  training  re¬ 
sources  available. 

In  response  to  this  need,  the  U.S.  Army  Research  Institute 
for  the  Behavioral  and  Social  Sciences  (ARI)  has  developed  an 
easy-to-use  method  for  predicting  how  rapidly  individual  tasks 
will  be  forgotten  over  intervals  of  no  practice  extending  up  to 
1  year.  The  method  has  been  developed  in  both  paper-and- pencil 
and  computer-based  format  and  is  geared  to  help  trainers  decide 
what  tasks  are  most  likely  to  be  forgotten,  how  many  soldiers 
will  be  able  to  perform  a  task  correctly  at  any  point  in  time, 
and  when  and  how  often  refresher  training  should  be  conducted. 

This  report  describes  the  results  of  research  performed 
over  the  past  3  years  to  support  development  of  this  method. 


Technical  Director 
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ACQUISITION  AND  RETENTION  OF  SOLDIERING  SKILLS 


EXECUTIVE  SUMMARY 


Requirement: 

To  develop  and  validate  a  convenient,  practical  method  that 
individual  unit  commanders  and  training  managers  can  use  when 
deciding  how  to  allocate  training  resources  in  order  to  maximize 
combat  readiness. 


Procedure : 

This  report  describes  the  third  and  final  year  of  a  3-year 
effort.  Previous  project  accomplishments  are  briefly  reviewed, 
and  activities  associated  with  the  third  year  of  effort  are  de¬ 
scribed  in  detail.  These  Year  III  activities  include:  (1)  a 
field  experiment  of  acquisition  and  retention  performance  of 
Field  Artillery  tasks,  using  soldiers  in  MOS  13B.  Approximately 
140  soldiers  were  trained  on  22  tasks  and  tested  for  recall  at 
2-month  intervals.  Also,  the  effects  of  overtraining,  previous 
testing,  and  soldier  abilities  (i.e.,  ASVAB  composite  scores) 
were  examined.  (2)  The  development  of  a  User's  Decision  Aid, 
which  uses  ratings  of  task  characteristics  to  estimate  retention 
functions  for  each  task.  (3)  The  assessment  of  the  relationship 
between  the  predicted  and  empirically  obtained  retention 
functions . 


Findings : 

Results  indicated  that  it  was  possible  to  estimate  soldiers' 
proficiency  accurately  over  time,  using  the  User's  Decision  Aid 
ratings.  Correlations  between  actual  retention  performance  and 
proficiency  levels  estimated  by  UDA  were  in  the  range  of  r  = 

0.90.  Soldier  abilities  and  overtraining  were  not  systematically 
related  to  performance. 


Utilization  of  Findings: 

Applications  of  the  User's  Decision  Aid  to  estimate  profi¬ 
ciency  levels  over  time  could  prove  useful  in  several  ways. 
Short-  and  long-term  scheduling  of  training  and  refresher  train¬ 
ing  could  be  improved.  The  User's  Decision  Aid  could  be  inte¬ 
grated  into  existing  Battalion  Training  Management  Systems. 
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INTRODUCTION 


A  major  function  of  the  Army  is  to  train  and  maintain 
combat- ready  troops.  Providing  effective  training  within 
resource  constraints  of  the  unit  requires  unit  commanders  and 
training  managers  to  answer  several  questions.  Which 
soldiers  need  training?  Which  tasks  need  to  be  trained?  When 
should  training  be  scheduled?  How  much  training  will  be 
required? 

Performance  on  any  task  declines  if  the  task  is  not 
practiced  periodically.  In  addition,  different  tasks  have 
different  rates  of  forgetting  (Schendel,  Shields,  &  Katz, 
1978;  Hagman  &  Rose,  1983).  Thus,  effective  training 
management  requires  identification  of  tasks  most  likely  to  be 
forgotten  and  those  least  likely  to  be  retained  in  order  to 
obtain  maximum  payoff  for  the  resources  spent. 

It  is  not  feasible  to  test  large  numbers  of  soldiers  on 
each  Army  task  to  find  out  the  rates  of  performance 
deterioration  associated  every  task.  However,  one  can  use 
theoretical  and  empirical  research  to  identify  categories  of 
tasks  most  likely  to  require  frequent  or  infrequent  training 
to  maintain  proficiency. 

Identification  of  these  task  characteristics  was  the 
goal  of  this  project.  Our  intent  was  to  produce  a 
convenient,  practical  method  that  individual  unit  commanders 
and  training  managers  could  use  when  deciding  how  to  allocate 
training  resources  in  order  to  maximize  combat  readiness.  In 
addition,  we  wanted  to  examine  the  effects  on  retention  of 
several  moderating  variables,  such  as  soldier  ability  and 
practice * 

Four  specific  objectives  of  the  project  were  identified: 

e  to  determine  task  characteristics  that 
influence  acquisition  and  retention; 

•  to  determine  the  effects  of  moderating 
variables  such  as  soldier  ability  and 
practice ; 

•  to  develop  a  method  for  predicting  individual 
task  retention;  and 

•  to  package  this  information  into  a  format 
that  Army  personnel  could  use  to  assess 
training  needs  and  to  increase  training 
effectiveness  within  the  unit. 
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Our  approach  for  meeting  these  specific  objectives 
consisted  of  several  tasks.  First,  we  reviewed  experimental 
and  theoretical  literature  concerning  acquisition  and 
retention  for  different  tasks,  soldiers,  and  time  intervals. 
Second,  we  developed  a  Task  Classification  System  (TCS)  that 
categorized  tasks  on  the  basis  of  task  dimensions  related  to 
retention.  This  TCS  formed  the  core  of  the  User's  Decision 
Aid  (UDA),  a  systematic  rating  procedure  and  algorithm  that 
generates  predictions  of  soldier  proficiency  levels  over 
time.  Third,  we  conducted  a  series  of  field  experiments  to 
obtain  data  for  validating  predicted  retention  loss  rates  for 
different  kinds  of  tasks  in  several  Military  Occupational 
Specialties  (MOSs). 

This  report  (a)  reviews  and  summarizes  the  tasks 
conducted  and  the  results  achieved  during  the  first  two  years 
of  the  project,  and  (b)  describes  in  detail  the  activities 
associated  with  the  project's  third  and  final  year. 


Summary  of  Year  I  and  Year  II  Results 


Year  I 

The  first  year  of  research  was  devoted  to  three  major 
tasks:  (1)  conducting  a  literature  review;  (2)  oeginning  the 

development  of  the  Task  Classification  System  (TCS);  and 
(3)  collecting  performance  data  in  the  field  on  a  sample  of 
military  tasks  (Rose  &  Ford,  1982). 

Literature  review.  Journal  articles  and  technical 
reports  pertaining  to  acquisition  and  retention  were 
reviewed,  with  the  focus  on  those  projects  conducted  or 
supported  by  the  Army  Research  Institute  ( ARI ) ,  using  Army 
tasks  performed  in  Army  settings. 

Several  of  the  ARI  projects  investigated  training 
variables  that  influence  task  retention.  The  results  of 
these  projects  showed  that:  (a)  overtraining  improved  task 
retention;  (b)  inclusion  of  additional  test  trials  and  the 
spacing  of  repetitions  during  training  were  effective  in 
promoting  retention;  and  (c)  innovative  instructional 
techniques  such  as  the  use  of  mnemonics  could  enhance 
retention  under  some  conditions. 

Task  characteristics  that  were  investigated  included 
difficulty,  interstep  cueing,  step  "relevance"  (i.e.,  its 
perceived  connection  to  the  task),  and  required  number  of 
steps.  These  characteristics  also  were  found  to  affect  task 
retention. 

Several  ARI  projects  addressed  the  question  of 
predicting  retention  from  individual  ability  measures  (e.g., 
Armed  Forces  Qualification  Test  (AFQT)  and  Armed  Services 


Vocational  Aptitude  Battery  [ASVAB]  scores).  While  the 
findings  from  these  projects  were  difficult  to  interpret, 
there  was  consensus  that  level  of  soldier  ability  was  related 
to  retention  of  skilled  performance. 

The  literature  review  had  several  implications  for  other 
segments  of  this  project.  It  directed  our  attention  to 
certain  variables  whose  relationship  to  acquisition  and 
retention  clearly  needed  further  empirical  study.  These 
variables  —  training  strategy,  individual  soldier 
differences,  task  characteristics,  and  practice  —  formed  the 
set  of  independent  variables  underlying  our  field  data 
collection  effort.  Furthermore,  the  literature  review  helped 
to  reveal  a  set  of  task  characteristics  presumed  to  be 
related  to  retention;  the  TCS  was  structured  around  many  of 
these  characteristics  (Rose,  McLaughlin,  Pelker,  &  Hagman, 
1981). 

Task  Classification  System.  During  Year  I,  we 
constructed  a  preliminary  TCS  based  on  the  results  of  the 
literature  review,  and  then  evaluated  it.  Construction  of 
the  TCS  involved  specifying  task  characteristics, 
operationalizing  their  definitions,  developing  rating  scales 
and  anchor  points,  and  determining  scale  weights  and 
combination  rules. 


At  the  end  of  Year  I,  the  TCS  contained  11  dimensions 
related  to  military  task  performance.  These  dimensions  were 
organized  into  three  general  categories; 


•  Enabling  Skills;  skills  that  are  adjunct  to 
the  task  but  that  enable  it  to  be  performed 
(e.g.,  “Use  of  Auxiliary  Equipment"); 


•  Task  Characteristics:  the  steps  required 
for  task  performance,  the  relationships 
among  steps,  and  the  information-processing 
requirements  of  steps  (e.g.,  "Number  of 
Steps");  and 


•  Criterion  Characteristics:  the  performance 
criteria  (e.g.,  "Consequences  of  Effort") 

(Rose,  Allen,  &  Johnson,  1982). 

Data  collection.  Data  were  collected  in  the  field  to 
determine  the  effects  of  variables  indicated  in  the 
literature  to  be  important  determinants  of  task  retention: 
length  of  no-practice  interval,  soldier  ability,  degree  of 
initial  learning,  and  task  characteristics.  In  our  initial 
field  work,  we  collected  acquisition',  retention,  and 
relearning  data  for  a  sample  of  tasks  performed  by  Track 
Vehicle  Mechanics,  MOS  63N. 
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The  general  approach  for  data  collection  was  to 
administer  and  score  hands-on  tests.  Each  mechanic  was 
tested  on  six  tasks  in  two  different  sessions.  During  the 
first,  session  (i.e.,  the  Acquisition  phase),  mechanics  were 
tested  and  scored  on  the  tasks,  given  feedback  regarding 
accuracy,  and  then  asked  to  repeat  the  task  until  they 
achieved  proficiency  (i.e.,  one  correct  performance).  They 
then  received  extra  training  on  half  the  tasks,  consisting  of 
two  extra  " test-f eedback-retest"  cycles;  that  is,  each 
soldier  was  tested  and  scored,  given  feedback,  and  retested 
until  he  performed  the  task  correctly  twice  more.  We  called 
this  “mastery"  training.  The  second  session  (i.e.,  the 
Retention  phase)  occurred  about  two  months  later.  The  same 
hands-on  tests  were  given  again  to  determine  retention. 
Following  the  retention  test,  mechanics  relearned  each  task 
back  to  proficiency  criterion  levels. 

The  general  picture  of  acquisition  and  retention 
performance  was  as  follows: 

(1)  Acquisition  -  Performance  was  scored  in  three  ways: 
number  of  steps  GO,  percentage  of  mechanics  GO  (i.e., 
performing  a  task  without  errors),  and  task  performance  time. 
We  assessed  the  acquisition  function  for  each  task:  number 
of  steps  GO  increased  for  all  tasks  from  the  first  to  the 
second  repetition,  where  almost  perfect  performance  was 
reached.  Similarly,  time  to  perform  each  task  decreased  with 
practice  during  the  first  session. 

(2)  Retention  -  At  the  Retention  phase,  there  was 
little  or  no  forgetting  in  terms  of  number  of  steps  performed 
correctly  for  any  task  after  two  months  of  no  practice. 
Performance  was  virtually  perfect,  with  a  mean  of  less  than 
one  error  per  task.  A  slightly  different  picture  emerged  for 
the  percentage-of-mechanics  GO  measure:  on  this  measure, 
there  were  retention  losses  for  some  tasks  for  both  the 
mastery  and  proficiency  groups  of  mechanics. 

Retention  performance  for  the  time  measure  showed 
forgetting:  time  to  complete  a  task  increased  substantially 

for  all  tasks.  This  retention  loss  was  not  systematic  across 
tasks;  while  performance  for  all  tasks  slowed  down  compared 
to  the  last  trial  of  Acquisition,  performance  on  some  tasks 
slowed  down  more  than  on  others. 

Retention  performance  was  not  related  systematically  to 
soldier  ability,  as  measured  by  ASVAB  or  AFQT  scores.  Also, 
retention  performance  was  not  related  systematically  to 
degree  of  original  learning  (i.e.,  proficiency  vs.  mastery). 
Furthermore,  there  were  no  interactions  among  the  major 
experimental  variables  (degree  of  original  learning,  ability, 
and  retention  interval). 
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With  regard  to  relearning,  there  were  no  systematic 
effects  related  to  ability  or  task  differences.  This  was  due 
to  lack  of  variance:  relearning  of  all  tasks  was  virtually 
complete  within  two  trials. 

The  key  to  understanding  these  results  lies  in  the  fact 
that  performance  in  the  63N  MOS  is  aided  by  technical  manuals 
( TMs ) .  Mechanics  used  these  manuals  while  performing  all 
tasks.  Thus,  all  a  mechanic  needed  to  do  was  follow  the 
manual  to  perform  the  tested  task.  The  sole  sources  of 
errors  would  be  ambiguities  or  deficiencies  in  the  TMs 
themselves  or  the  mechanic's  lack  of  familiarity  with 
specific  tools. 

One  further  assumption  accounted  for  practically  all  of 
the  results:  performance  of  63N  tasks  is  normally  untimed 
(i.e.,  there  are  no  prescribed  time  limits).  We  speculated 
that  mechanics  "took  their  time  and  did  it  right"  during  the 
retention  test,  thus  accounting  for  the  slower  performance 
t imes . 

These  results  did  not  shed  much  light  on  variables 
affecting  acquisition  and  retention  of  skills,  other  than  to 
indicate  the  importance  of  job  aids.  Thus,  the  focus  of 
Year  II  data  collection  was  on  acquisition  and  retention  of 
tasks  that  were  not  job  aided. 


Year  II:  Data  Collection 

A  major  activity  during  Year  II  was  the  collection  of 
acquisition  and  retention  data  for  a  sample  of  18  MOS  1IB10 
(Infantryman)  tasks  (Rose,  Manning,  Radtke,-  &  Ford,  1983). 

We  selected  tasks  meeting  four  criteria:  the  task  was 
contained  in  the  11B  Skill  Level  1  Soldier's  Manual  (SM);  the 
task  was  riot  going  to  be  included  in  the  11B  Skill 
Qualification  Test  ( SQT )  hands-on  component,  SQT  skill 
component,  or  Expert  Infantry  Badge  (EIB)  tests;  the  logistic 
demands  for  the  task  could  fit  our  administrative 
requirements;  and  the  tasks  would  sample  a  range  of  task 
characteristics  identified  in  the  TCS. 


We  developed  a  hands-on  test  for  each  of  the  18  tasks. 
Each  test  consisted  of  a  scoresheet  and  administrative 
instructions.  Administrative  instructions  followed  the  same 
format  as  SQT  hands-on  tests. 


Data  collection.  Data  were  collected  in  four  phases. 
During  the  first  "Acquisition"  phase,  165  soldiers  were 
rested  and  trained  on  all  tasks.  Hands-on  tests  covering  the 
selected  tasks  were  administered  first  during  the  Acquisition 
phase.  If  a  soldier  failed  a  task,  the  scorer  told  him  each 
step  performed  incorrectly  and  demonstrated  steps  if 
necessary.  After  this  training,  the  soldier  performed  the 
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task  again,  and  the  process  was  repeated  until  he  performed 
the  task  without  error. 


After  being  tested  and  trained  on  all  tasks,  each 
soldier  received  "Mastery"  training  on  half  of  the  tasks. 
That  is,  after  completing  all  the  tasks  once  correctly,  each 
soldier  returned  to  half  of  the  stations,  where  he  repeated 
each  task  until  it  was  performed  without  error;  this  "repeat 
cycle"  was  done  twice  so  that  each  soldier  completed  three 
errorless  performances  on  half  of  the  tasks. 


During  the  second  phase,  about  two  months  later,  we 
tested  one-third  of  the  soldiers  on  all  the  tasks.  During 
the  third  phase,  four  months  after  Acquisition,  we  tested  the 
soldiers  from  Phase  Two  and  another  one-third  of  the  original 
group.  Soldiers  tested  at  the  two-month  and  four-month 
retention  intervals  performed  each  task  one  time  with  no 
assistance  or  feedback.  Scorers  were  told  not  to  give 
soldiers  any  information  about  their  overall  performance 
("GO"  or  "NO-GO")  or  about  specific  steps.  During  Phase 
Four,  six  months  after  Acquisition,  we  tested  and  retrained 
all  soldiers  in  the  project.  No  retraining  occurred  during 
the  second  and  third  phases. 


Scorers  conducted  the  six-month  retention  test  the  same 
way  they  conducted  the  initial  Acquisition  test.  If  the 
soldier  made  a  mistake,  the  scorer  told  him  what  the  mistake 
was  and  had  him  do  the  task  over.  The  scorer  also  recorded 
the  number  of  trials  the  soldier  needed  before  he  was  able  to 
do  the  task  without  error. 


Results.  In  general,  soldiers  did  well  on  first-trial 
acquisition  performance  measures  on  tasks  requiring  primarily 
physical,  as  opposed  to  primarily  mental  or  verbal  activity. 
Soldiers  did  not  do  as  well  on  tasks  that  required  processing 
of  verbal  information,  performance  of  complex  procedures,  or 
decision  making.  More  trials  were  typically  required  to 
learn  these  latter  tasks. 


From  an  analysis  of  the  retention  data,  all  tasks  showed 
forgetting  after  two  months.  Retention  losses  for  mental 
tasks  were  dramatic.  During  the  two-month  test,  no  soldier 
could  correctly  name  vehicles,  and  very  few  could  identify 
vehicles  as  friend  or  foe.  On  the  other  hand,  performance 
was  quite  good  on  some  physical  tasks.  Between  the  two-month 
and  four-month  Phases,  performance  declined  on  most  of  the 
mental  tasks,  while  for  most  physical  tasks,  performance 
remained  about  the  same  between  the  two  phases. 


The  descriptive  statistics  for  the  six-month  retention 
test  showed  a  dramatic  increase  in  proficiency  on  practically 
all  tasks  when  compared  to  the  two-  and  four-month  retention 
scores.  Since  it  was  clear  that  these  data  did  not  reflect 
forgetting,  we  elected  not  to  conduct  any  detailed  analyses 


of  them.  (The  increased  proficiency  was  attributed  to  a 
variety  of  procedural  artifacts  introduced  by  the 
participating  unit  (see  Rose,  et  al.,  ibid.]). 

Soldier  abilities.  We  conducted  an  analysis  to  assess 
whether  individual  abilities  of  soldiers,  as  measured  by  the 
ASVAB ,  were  related  to  performance.  We  found  no  systematic 
relationships  between  a  single  composite  score  or  a  set  of 
composite  scores  and  performance  on  a  large  number  of  tasks. 
Rather,  for  some  of  the  tc'sks,  all  of  the  composite  scores 
seemed  to  be  correlated  with  performance  fairly  well,  while 
for  other  tasks,  performance  was  not  related  at  all  to  ASVAB 
scores.  Furthermore,  we  could  not  detect  any  commonalities 
among  the  tasks  (and  measures)  that  were  or  were  not 
correlated  with  ASVAB. 

/ 

Year  II:  The  User's  Decision  Aid  (UDA) 

UDA  development.  During  Year  II,  a  major  goal  of  the 
project  was  to  begin  development  of  a  convenient,  practical 
method  that  individual  unit  commanders  and  training  managers 
could  use  when  deciding  how  to  allocate  training  resources 
based  on  task  retention  predictions.  The  core  of  this  UDA 
would  consist  of  a  way  to  estimate  or  predict  a  unit's  level 
of  proficiency  for  any  given  task  at  any  given  point  in  time 
—  that  is,  an  algorithm  that  generates  "pure"  task  retention 
functions.  The  basis  for  such  an  algorithm  was  the  TCS 
described  above  in  the  review  of  Year  I  research. 

During  Year  II,  we  identified  certain  task  dimensions 
that  were  most  likely  to  be  related  to  retention.  We  then 
converted  these  dimensions  into  rating  scales,  developed 
anchor  points,  and  analytically  assigned  weights  to  each 
point  on  the  scales.  Next,  we  assessed  each  scale's 
reliability  and  validity  by  having  several  judges  rate  tasks 
on  each  scale.  We  examined  both  interrater  agreement  and  the 
correlation  between  task  ratings  and  actual  retention  data. 
These  steps  were  repeated  —  dimensions  were  redefined, 
different  weights  chosen,  new  dimensions  added  —  several 
times  in  order  to  develop  a  set  of  rating  scales  for  the  UDA. 

We  also  worked  on  field  implementation  of  the  UDA  — *  how 
it  should  be  presented  to  the  user,  how  judgments  should  be 
recorded  and  processed,  and  how  the  resulting  predictions  and 
estimates  of  performance  should  be  displayed.  We  developed 
two  different  versions  of  the  UDA  for  possible 
implementation,  depending  upon  potential  resources  available 
in  the  field.  The  first  was  a  computer  program,  where  users 
interact  directly  with  the  program  via  a  computer  keyboard. 
The  second  was  a  paper-and-penci 1  version,  where  users  record 
their  responses  and  manually  compute  task  retention 
estimates . 


The  heart,  of  the  UDA  ic  an  algorithm  that  weights  and 
summarizes  the  relevant  characteristics  of  a  task  to  produce 
a  single  task  retention  "score."  This  score  is  used  to 
predict  the  level  of  task  proficiency  of  a  unit  over  time. 

Users  rate  tasks  by  answering  a  series  of  questions 
posed  by  the  UDA.  Their  answers  are  combined  by  the 
algorithm  to  produce  both  a  retention  score  and  a  projected 
rate  of  proficiency  loss. 

UDA  evaluation.  A  preliminary  examination  of  the  UDA 
algorithm  revealed  several  important  findings.  First,  the 
interrater  reliability  was  relatively  high  (e.g.,  r  =  .90+). 
Second,  the  algorithm  appeared  to  tap  characteristics  of 
tasks  that  are  relevant  to  the  training  of  military 
personnel.  The  summary  retention  scores  and  the  component 
ratings  correlated  significantly  with  retention  performance 
measures  of  accuracy  and  time. 

The  evaluation  also  highlighted  the  directions  that 
further  development  should  take.  The  algorithm  needed 
further  testing  and  refinement  in  four  areas.  First,  terms 
used  in  the  algorithm  required  more  complete,  unambiguous 
definitions.  Second,  the  instrument  needed  to  be  tested  with 
a  greater  number  of  tasks  having  more  diverse 
characteristics.  Third,  the  number  and  variety  of  raters 
using  the  algorithm  needed  to  be  increased.  Finally,  the 
algorithm  needed  to  be  examined  in  relation  to  retention  over 
longer  periods  of  time  (e.g.,  2-6  months). 

Year  II:  Predicting  Performance 

Regression  analyses.  An  important  practical  question 
for  this  project  is  whether  acquisition  and  retention 
performance  can  be  predicted  from  other  information.  During 
Year  II,  we  employed  two  types  of  regression  analyses  to 
predict  performance.  The  first  type  used  individual 
difference  variables  (e.g.,  ASVAB  scores)  to  predict 
soldiers'  performance  separately  for  each  task.  The  second 
analysis  used  task  difference  and  group  difference  variables 
to  predict  overall  task  performance:  that  is,  differences 
among  the  entire  set  of  tasks. 

In  the  first  set  of  regression  analyses,  we  wanted  to 
see  whether  measures  of  individual  ability  and  frequency  and 
recency  of  task  performance  would  predict  first-trial 
Acquisition  performance.  For  first-trial  Acquisition  scores, 
the  analyses  suggested  that  soldiers'  individual  differences 
did  not  seem  to  predict  their  performance  on  a  given  task. 

In  a  second  set  of  regression  analyses,  however,  we  found 
that  information  about  group  and  task  characteristics  could 
be  used  to  predict  group  Acquisition  performance  on  a  set  of 
tasks. 
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We  performed  multiple  regression  analyses  similar  to 
those  used  to  analyze  the  first-trial  Acquisition  data  to  see 
if  individual  difference  variables  or  group  and  task 
variables  could  predict  two-month  retention  performance. 

For  most  tasks,  our  regression  equations  based  on 
individual  difference  variables  accounted  for  a  larger 
proportion  of  the  variance  in  two-month  retention  performance 
than  they  could  account  for  when  predicting  Acquisition 
performance.  Prediction  of  performance  for  the  mental  tasks 
was  fairly  good.  Our  individual  difference  variables  did  not 
predict  performance  very  well  on  the  physical  tasks. 

However,  task  variables,  especially  the  UDA  ratings, 
demonstrated  substantial  correlations  with  two-month 
retention  performance:  correlations  ranged  between  r  =  0.80 
and  r  =  0.90,  depending  upon  the  dependent  measure  chosen  to 
characterize  performance. 

We  performed  regression  analyses  to  determine  whether 
four-month  retention  performance  could  be  predicted  by 
individual  difference  variables  and  by  group  and  task 
variables.  Prediction  of  four-month  retention  performance 
was,  for  the  most  part,  not  quite  as  good  as  prediction  of 
two-month  retention  performance.  No  more  than  about  25%  of 
the  variance  in  any  performance  measure  was  accounted  for  by 
the  set  of  individual  difference  predictors.  However,  the 
UDA  ratings  were  again  highly  correlated  with  retention 
performance . 

The  primary  implication  of  these  results  is  that  it  is 
possible  to  estimate  soldiers’  proficiency  fairly  accurately 
over  time.  For  certain  performance  measures  (i.e., 
"Percentage  of  soldiers  GO"  and  "Percentage  of  steps  GO"), 
excellent  estimates  of  proficiency  could  be  obtained  from  the 
task  ratings  produced  by  the  UDA. 


II.  YEAR  Ills  ACQUISITION  AND  RETENTION 
OF  CANNON  CREWMAN  TASKS 


A  major  activity  during  Year  III  was  the  collection  of 
acquisition  and  retention  data  for  a  sample  of  13B10 
(Cannon  Crewman)  MOS  tasks.  The  goals  of  this  activity 
were : 

•  to  empirically  establish  the  acquisition 
and  retention  functions  for  a  variety  of 
Cannon  Crewman  tasks; 

•  to  continue  exploration  of  the 
relationships  of  soldier  abilities,  level 
of  initial  learning,  task  characteristics, 
and  retention  interval  to  retention;  and 

•  to  provide  criterion  data  for  the 
assessment  of  the  validity  of  the  UDA. 


Method 


Selection  of  MOS 

The  MOS  for  Year  III  was  13B10,  Cannon  Crewman.  We 
selected  this  MOS  because  it  had  begun  implementing  the 
Cohesion  Operational  Readiness  and  Training  (COHORT) 
program,.  Under  this  program,  all  soldiers  in  a  One  Station 
Unit  Training  (OSUT)  battery  are  assigned  as  a  group  to  the 
same  unit  and  stay  in  that  unit  throughout  their  first  tour 
of  duty.  Our  experience  with  an  Infantry  COHORT  company 
during  Year  II  had  confirmed  our  expectation  that  the 
reduction  in  personnel  turbulence  in  the  COHORT  program 
would  enhance  the  feasibility  of  a  longitudinal  retention 
design . 


Selection  of  Tasks 

The  initial  domain  of  tasks  consisted  of  all 
13B-specif ic ,  Skill  Level  one  (SL1)  tasks  .listed  in 
Soldier's  Manual  FM  6-13B  (U.S.  Army,  1982).  We  then 
narrowed  the  domain  to  tasks  that  were  performed  by  the 
supporting  battalion  at  Fort  Ord,  California.  Since  the 
battalion  was  assigned  to  the  M198  (Towed)  Howitzer,  we 
excluded  tasks  performed  on  other  howitzers. 
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We  also  excluded  tasks  that  pertained  to  restricted 
duty  positions,  specifically  Vehicle  Driver,  Artillery 
Mechanic,  Assembler:  155-MM  Atomic  Projectile,  Demolitions, 
and  Nuclear  Security,  Our  rationale  for  these  exclusions 
was  twofold:  First,  we  anticipated  serious  support 
problems  since  qualified  personnel  and  equipment  necessary 
to  test  and  train  these  tasks  would  not  be  available  from 
the  supporting  unit.  Second,  we  wanted  the  battalion  to 
perceive  their  participation  in  the  project  as  beneficial 
to  all  duty  positions.  Finally,  we  excluded  tasks  from  the 
initial  domain  that  were  similar  to  Infantry  tasks  tested 
in  Year  II.  We  therefore  did  not  consider  Navigation, 
Conununicat ion ,  or  Combat  Tactics  tasks. 

The  resulting  domain  for  task  selection  consisted  of 
the  26  tasks  listed  in  Table  1.  We  gave  priority  to  tasks 
that  had  been  trained  in  OSUT  and  tested  in  the  End  of 
Course  (EOC)  test  in  OSUT.  We  wanted  to  select  a  set  of 
tasks  that  would  include  a  wide  range  of  initial 
proficiency  levels.  Therefore,  we  used  the  percent  of  SL1 
soldiers  in  the  Army  Occupational  Survey  Program  (AOSP) 
survey  who  said  they  had  performed  the  task  as  an  indicator 
of  proficiency:  we  expected  that  the  tasks  performed  by 
smaller  numbers  of  incumbents  would  be  associated  with 
lower  initial  test  scores. 

The  final  consideration  was  to  be  sure  that  the  set  of 
tasks  selected  could  be  tested  with  che  resources  of  the 
supporting  battalion.  Those  resources  dictated  that  at 
least  30  soldiers  had  to  be  tested  per  day,  using  no  more 
than  one  battery's  equipment.  We  identified  a  set  of 
eleven  tasks  that  could  be  tested  within  the  time  and 
equipment  constraints.  Those  tasks  are  marked  "Selected" 
under  REMARKS  in  Table  1. 

After  we  had  developed  the  initial  drafts  of  the  tests 
of  the  eleven  tasks,  we  tried  them  out  with  the  supporting 
battalion.  The  tryout  and  the  results  are  discussed  later 
under  Test  Development;  part  of  the  tryout,  however, 
affected  task  selection.  Before  the  tryout,  four 
noncommissioned  officers  (NCOs),  who  were  to  serve  as 
scorers,  reviewed  the  tests.  As  a  result  of  this  review, 
the  test  of  Perform  Preventive  Maintenance  Checks  and 
Services  (PMCS)  was  deleted  and  replaced  by  the  Perform 
Gunner's  Quadrant  Micrometer  Test.  This  is  by  doctrine  a 
SL3  task,  but  it  is  relevant  to  the  M198  SL1  because  the 
quadrant  must  be  used  to  boresight. 


Table  1 

Summary  of  Review  of  13B  (M198)  SL  1  Tasks 


TRAINED 

TESTED 

A0SP 

CANNONEER 

0SUT 

EOC 

FREQ 

POSITION 

REMARKS 

COMMON  MOS  TASKS  (SL1) 

061-266-1101  Prepare  a  position 
to  receive/emplace 
a  howitzer 

YES 

NO 

48 

(As  Assigned) 

Selected 

061-266-1102  Record/maintain  fire 
mission  data  on  DA 
Form  4513  (Record  of 
Missions  Fired) 

NO 

NO 

12 

(As  Assigned) 

061-266-1103  Emplace/recover 
collimator 

YES 

YES 

70 

#3  or  4 

Selected 

061-266-1104  Emplace/recover 
aiming  posts 

YES 

YES 

66 

//3  or  4 

Selected 

AMMUNITION  (SL1) 

061-266-1501  Load  howitzer 
ammunition  on 
vehicles 

YES 

NO 

45 

(As  Assigned) 

061-266-1504  Store  ammunition  in 
preparation  for 
firing 

YES 

NO 

35 

All 

No  inert 
ammunition  in 
unit 

CREW  SERVED  WEAPONS  (SL1) 

071-312-3005  Perform  operator 
maintenance  on  an 

M60  machinegun  and 
ammunition 

YES 

YES 

46 

(As  Assigned) 

Selected 

071-312-3007  Prepare  a  range 
card  for  an  M60 
machinegun 

YES 

NO 

18 

(As  Assigned) 

071-313-3451  Perform  operator 
maintenance  on  a 
caliber  .50  HB 
machinegun  and 
ammunition 

NO 

NO 

33 

(As  Assigned) 

Cal  .50  not 
primary 
weapon  of 

Light  Inf 
Division 

Table  1  (continued) 


TRAINED  TESTED  AOSP  CANNONEER 

OSUT  EOC  FREQ  POSITION  REMARKS 

071-313-3453  Load,  reduce  a  YES  NO  21  (As  Assigned) 

stoppage,  unload, 
and  clear  a  caliber 
. 50  machinegun 

071-313-3454  Engage  targets  with  YES  NO  21  (As  Assigned) 

a  caliber  .50 
machinegun 

071-313-3455  Set  headspace  and  YES  NO  32  (As  Assigned) 

timing  on  a  caliber 
.50  machinegun 

071-313-3461  Mount/dismount  a  YES  NO  34  (As  Assigned) 

caliber  .50  HBM2 
machinegun  on  a 
tripod 

071-313-3462  Mount/dismount  a  YES  NO  34  (As  Assigned) 

caliber  .50  HBM2 
machinegun  on  a 
tracked  vehicle 


(M1J8)  CANNONEER  (SL1) 

061-266-1506  Prepare  separate  YES  (SP  Only)  62  All  Selected 

loading  ammunition 
for  firing 


061-271-1212  Boresight  the  direct  YES  NO  35  Assistant  Selected 


fire  telescope  using 
a  distant  aiming 
point  (DAP) 

Gunner 

061-271-1215 

Set /lay  the  cannon 
for  quadrant  with 
the  range  quadrant 

YES 

NO 

30 

Assistant 

Gunner 

Selected 

061-271-1216 

Measure  the  quadrant 
with  the  range 
quadrant 

YES 

NO 

28 

Assistant 

Gunner 

Selected 

061-271-1217 

Sight  on  a  target 
with  the  direct  fire 
telescope  (M198) 

YES 

NO 

29 

Assistant 

Gunner 

Selected 
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Table  1  (continued) 


TRAINED 

TESTED 

A0SP 

CANNONEER 

0SUT 

EOC 

FREQ 

POSITION 

REMARKS 

061-271-1410 

Disassemble/assemble 

YES 

NO 

28 

(As  Assigned) 

Selected 

breech  and  firing 
mechanism  (M198) 

061-271-1507 

Load  and  fire  a 
prepared  round 

YES 

YES 

40 

All 

Crew  drill. 

M712  AMMUNITION  (SL1) 

061-266-1509 

Inspect  the  M712 

NO 

NO 

N/A 

All 

CONUS  units 

projectile 

did  not  have 
the  M712 
training 

061-266-1510 

Repackage  the  M712 
projectile 

NO 

NO 

N/A 

All 

proj ectile 

061-266-1511 

Perform  extraction 
procedures  on  M712 
proj ectile 

NO 

NO 

N/A 

All 

061-266-1512 

Unpackage  and  pre- 

NO 

NO 

N/A 

All 

pare  the  M712 
projectile 


Ml  98)  CANNON  MAINTENANCE  (SL1) 

061-271-1601  Perform  preventive  NO  NO  15  All  Selected 

maintenance  checks 
and  services  (PMCS) 

(M198) 


The  final  eleven  tasks  fell  into  three  groups.  The 
tasks  selected  by  group  were: 

•  Individual  tasks  —  perform  operator 
maintenance  on  the  M60  machinegun, 
disassemble/assemble  breech  mechanism, 
prepare  a  position  to  receive/emplace  a 
howitzer  ? 

•  Crew  tasks  —  Emplace/recover  aiming 
posts,  emplace/recover  collimator,  prepare 
separate  loading  ammunition  for  firing; 
and 

•  Advanced  tasks  —  Measure  quadrant  with 
range  quadrant,  set/lay  cannon  for 
quadrant  with  range  quadrant,  sight  on  a 
target  with  direct  fire  telescope, 
gunner's  micrometer  test,  and  boresight 
direct  fire  telescope  using  Distant  Aiming 
Point  method. 


Test  Development 

Wv*  developed  a  hands-on  test  for  each  selected  task, 
following  the  format  for  SQT  hands-on  tests.  Each  test 
included : 

•  an  equipment  list  that  specified  equipment 
required  at  the  station, 

•  instructions  to  the  scorer  on  how  to  set 
up  the  test  site, 

•  instructions  to  the  scorer  on  how  to 
standardize  test  conditions, 

•  instructions  to  the  scorer  on  how  to 
evaluate  performance  not  specified  on  the 
scor.  sheet. , 

•  instructions  for  the  scorer  to  read  to 
soldiers  to  begin  the  test,  and 

•  scoring  points  that  the  3corer  was  to  mark 
GO  or  NO  GO  for  each  soldier. 


The  test  approaches  are  summarized  in  Table  2.  The 
complete  tests  for  the  selected  tasks  are  in  Appendix  A. 

Our  final  step  in  test  development  was  to  conduct  a 
check  of  scoring  procedures  for  each  test.  In  these 
tryouts,  four  NCO  (E5  or  E6)  scorers  were  given  the  various 
instructions  described  above,  and  each  scorer  performed 
each  task.  They  then  simultaneously  and  independently 
evaluated  five  SLl  soldiers  on  each  task.  The  scorers  and 
tested  soldiers  were  from  the  supporting  battalion. 

The  scorers  administered  the  tests  to  each  of  the  five 
SLl  soldiers.  After  each  soldier  finished  a  given  test, 
the  scorers  discussed  their  GO/NO  GO  ratings.  Project 
staff  resolved  the  disagreements  through  discussion  of 
doctrine  governing  the  task  or  revision  of  the  scoresheets. 
Scorers  did  not  change  the  rating  of  the  soldier  they  had 
just  evaluated,  but  they  did  modify  scoring  of  later 
soldiers. 

Although  the  emphasis  in  these  tryouts  was  on  the 
qualitative  evaluation  of  the  scoring  procedures, 
quantitative  analyses  were  also  conducted  to  estimate  the 
extent  to  which  raters  could  reliably  judge  soldier 
performance.  Using  generalizability  theory  (Brennan, 

1983),  an  approach  based  on  analysis  of  variance,  we 
analyzed  the  results  of  the  tryouts.  This  approach 
produces  an  index  called  a  generalizability  coefficient 
which  indicates  the  accuracy  of  scoring  procedures.  These 
coefficients  range  from  0.00  to  1.00.  The  closer  to  1.00, 
the  more  accurate  the  raters.  The  approach  produces  the 
index  in  a  way  which  allows  reasons  for  the  raters' 
inconsistencies  in  scoring  to  be  determined  when  the 
generalizability  coefficient  is  low,  e.g.,  less  than  0.60. 

Generalizability  analyses  produce  estimates  of  why 
test  scores  vary.  In  this  case,  test  scores  may  differ 
because  (1)  the  soldiers  being  tested  differ;  (2)  the 
raters  giving  the  scores  differ  (e.g.,  some  raters  are 
consistently  hard  and  others  easy);  or  (3)  the  raters  are 
inconsistent  (i.e.,  each  rater  randomly  shifts  from  being 
an  easy  rater  to  being  a  hard  rater).  The  second  and  third 
sources  of  influence  in  the  scores  obviously  make  the 
scores  inaccurate.  The  smaller  those  sources  of  variation 
compared  to  the  variation  among  soldiers,  the  more  accurate 
the  scores  are.  Table  3  presents  the  results  including 
generalizability  coefficients  and  estimates  of  the 
influence  of  the  two  kinds  of  rater  error  on  the  test 
scores. 


Table  2 

Summary  of  Approaches  for  Hands-On  Tests 


Prepare  a  Position  to  Receive/Emplace  The  soldier  acted  as  a  gun  guide.  He 
a  Howitzer  installed  the  pantel  and  gun  guide 

stakes,  set  up  a  field  telephone,  and 
recorded  deflection  from  the  aiming 
circle  and  meters  from  the  aiming 
circle  to  the  pantel  stake.  Because 
of  differences  among  SOP,  distance 
between  stakes  was  not  scored. 


Emplace/Recover  Collimator  A  scorer’s  assistant  operated  the 

gunner’s  telescope  while  the  tested 
soldier  set  up  the  collimator.  Since 
there  was  a  difference  among  the 
Batteries'  SOP  regarding  the  location 
of  the  collimator,  scorers  accepted  a 
wider  range  of  positions  than  antici¬ 
pated. 


Emplace /Recover  Aiming  Posts 


Perform  Operator  Maintenance  on  the 
M60  Machlnegun 


Prepare  Separate  Loading  Ammunition 
for  Firing 


The  howitzer  was  positioned  in  loca¬ 
tion  that  allowed  at  least  100  meters 
of  level  terrain.  The.  instructions 
gave  a  situation  that  required 
installation  of  night-lighting 
devices.  Soldiers  had  to  install 
night-lighting  devices,  position  the 
near  post  at  about  50  meters,  position 
the  far  post  at  about  100  meters  and 
adjust  the  near  post  following  scorer's 
hand  and  arm  signals.  The  scorer 
evaluated  the  distance  by  counting 
steps  to  the  position  (33-36) . 

The  soldier  had  to  clear  the  M60 
(without  cue),  disassemble  the  M60, 
assemble  the  M60,  and  perform  a 
function  check. 

Because  there  were  no  inert  rounds 
available,  soldiers  selected  from  color 
drawings  of  rounds,  fuzes,  and  powder. 
The  drawings  of  the  rounds  did  not 
include  the  ammunition  type.  The 
scorer  gave  five  fire  commands.  For 
each  command,  the  soldier  recorded  the 
command  on  a  Record-of-Fire  Form,  then 
indicated  the  round  (based  on  accurate 
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Table  2  (continued) 


Task 


Boresight  the  Direct  Fire  Telescope 
Using  a  Distant  Aiming  Point  (DAP) 
(M198) 


Perform  Gunner's  Quadrant  Micrometer 
Test 


Set/Lay  Cannon  for  Quadrant  with  the 
Range  Quadrant  (M198) 


Measure  the  Quadrant  with  the  Range 
Quadrant  (Ml 98) 


_ Approach _ _ 

markings  and  approximate  color) ,  and 
indicated  the  correct  fuze.  For  two 
commands  the  soldier  set  an  inert  fuze. 
For  one  command  the  soldier  also 
indicated  the  powder  charge  required. 

This  test  was  more  complex  for  the 
Ml  98  than  it  would  have  been  for  other 
howitzers.  Because  of  a  quirk  in  the 
M198  center  vent,  the  soldier  had  to 
adjust  the  lay  of  the  tube  2.3  mils 
before  aligning  the  telescope  with  the 
gun  tube.  The  adjustment  required  use 
of  the  gunner's  quadrant  and  recall  of 
the  amount  to  subtract.  There  was  a 
two  minute  time  limit. 

The  soldier  set  the  micrometer  to  zero 
and  the  index  arm  to  +10,  put  the 
quadrant  on  the  quadrant  seats, 
elevated  or  lowered  the  tube  until  the 
quadrant  bubble  was  level,  reversed 
the  settings,  and  rechecked.  There 
was  a  one  minute  time  limit. 

The  scorer  announced  a  quadrant.  The 
soldier,  acting  as  assistant  gunner 
(AG) ,  repeated  the  command ,  set  the 
quadrant,  centered  the  longitudinal 
bubble  by  elevating  or  depressing  the 
tube,  centered  the  cross-level  bubble, 
and  announced  SET.  There  was  a  30 
second  time  limit. 

The  scorer  measured  the  quadrant  of 
the  tube  then  off-set  the  longitudinal 
and  cross- level  bubbles.  The  soldier, 
acting  as  AG,  had  to  center  the  bubbles 
and  announce  the  quadrant  within  15 
seconds.  The  command  was  "Measure  the 
quadrant."  After  the  training  phase 
some  soldiers  reported  that  they  were 
used  to  the  command  "Level  the 
bubbles. " 


Table  2  (continued) 


Task 


Sight  on  a  Target  with  the  Direct 
Fire  Telescope  (M198) 


Disassemble/Assemble  Breech  and 
Firing  Mechanism  (M198) 


_ Approach _ 

A  target  was  positioned  at  a  range  of 
about  200  meters  (the  limit  for  the 
test  site).  The  scorer  announced  a 
direct  fire  command  with  the  range 
element  in  meters.  The  soldier  had  to 
repeat  the  range,  convert  the  range  to 
mils  (from  a  conversion  plate) ,  lay  the 
appropriate  reticle  point  on  the 
target,  and  repeatedly  announce  SET. 
There  was  a  30  second  time  limit. 

The  soldier  disassembled  the  breech 
mechanism,  assembled  the  breech,  and 
performed  a  function  check  (unc.ued). 

The  operator's  TM  was  available  and 
soldiers  were  scored  NO-GC  if  they  did 
not  open  the  TM.  For  the  acquisition 
phase,  soldiers  had  to  open  co  the 
correct  section  before  proceeding,  but 
they  rarely  referred  to  the  TM  after 
that. 


The  Mean  column  shows  the  level  of  performance  on  the 
complete  task.  It  is  the  number  of  GO  performance  measure 
ratings  from  the  four  scorers  for  the  five  tested  soldiers 
divided  by  the  total  number  of  performance  measure  ratings 
for  the  task. 

The  Rater  Main  Effect  column  shows  the  extent  to  which 
scores  varied  because  some  scorers  tended  to  be  stricter  or 
more  lenient  than  others.  No  evidence  of  consistent 
differences  among  scorers  emerged.  Therefore,  the  use  of 
different  scorers  for  different  soldiers  during  data 
collection  would  have  had  no  effect  on  the  relative 
standing  of  soldiers. 

The  Interaction  Effect  column  shows  the  extent  to 
which  any  scorer  was  inconsistent.  Compared  to  the 
estimates  of  person  differences,  the  effects  were 
negligible  except  for  two  tests  —  Perform  Gunner's 
Quadrant  Micrometer  Test  and  Sight  on  Target. 

The  last  column  shows  the  Generalizabil ity 
Coefficients  for  each  test.  These  indicate  how  accurate 
test  scores  would  be  if  different  scorers  were  used  to 
score  the  tests.  The  numbers  were  derived  by  dividing 
person  variance  by  the  sum  of  person  variance  plus  both 
error  variance  estimates. 

For  the  most  part,  the  generalizability  coefficients 
shown  in  Table  3  are  high.  There  were,  however,  two  tasks 
with  coefficients  below  0.60  —  Prepare  Position  and  Sight 
on  Target.  For  Prepare  Position,  the  raters'  interaction 
error  was  nearly  half  the  size  of  true-person  differences. 
This  resulted  from  disagreements  among  raters  regarding 
unit  SOP  and  test  doctrine.  We  modified  the  scoring  points 
to  accommodate  those  differences.  We  were  not  able  to 
identify  the  source  of  problems  with  Sight  on  Target.  The 
scoring  points  that  had  disagreements  were  both  oral 
responses  —  "Repeated  the  announced  range"  and  "Repeatedly 
announced  SET."  Scoring  of  both  should  have  been  reliable. 
The  most  likely  explanation  was  that  the  similarity  of  the 
soldiers  magnified  the  scorers'  unfamiliarity  with  the 
doctrine.  The  test  was  retained  with  the  expectation  that 
greater  familiarity  with  the  tasx  by  the  scorers  and  tested 
soldiers  would  result  in  acceptable  reliability  for  this 
task. 


Table  3 

Interrater  Reliability 


Mean 

Variance 

Generalizability 

Coefficient 

Task 

Person 

Rater 

Main  Effect 

Interaction 

Prepare  Position 

-69 

.003 

.000 

.002 

.56 

Emp  lace  /  Rec  ov  er 
Collimator 

.96 

.001 

.000 

.000 

.93 

Emplace/Recover 

Aiming  Posts 

.78 

.019 

.000 

.003 

.85 

Perform  Operator 
Maintenance  on  M60 

.93 

.001 

.000 

.000 

.99 

Prepare  Ammunition 

.84 

.018 

.000 

.000 

.99 

Boresight  Direct  Fire 
Telescope 

.85 

.012 

.000 

.002 

.89 

Perform  Gunner's  Quadrant 
Micrometer  Test 

Set/Lay  Gannon  for 
Quadrant 

Measure  Quadrant 

Sight  on  Target  with 
Direct  Fire  Telescope 

Disassemble/Assemble 
Breech 


57 

.070 

.001 

.036 

.65 

91 

.008 

.000 

.003 

.70 

57 

.020 

.000 

.007 

.73 

15 

.007 

.000 

.022 

.25 

90 

.001 

.000 

.000 

1.00 

Several  tasks  in  this  experiment  are  made  up  of  more 
than  one  major  subtask.  For  example,  "Perform  Operator 
Maintenance  on  the  M60  Machinegun"  has  four  distinct  parts, 
with  different  task  characteristics.  To  determine  whether 
these  subtasks  had  different  acquisition  and  retention 
functions,  we  subdivided  five  of  the  original  tasks  into 
sixteen  smaller  units,  each  dealing  with  a  specific 
procedure : 

Emplace/recover  collimator: 

•  Emplace  collimator 

•  Recover  collimator 

Prepare  separate  loading  ammunition  for  firing: 

•  Select  ammunition 

•  Select  powder 

•  Set  fuze 

Perform  operator  maintenance  on  the  M60  machinegun: 

•  Clear 

•  Disassemble 

•  Assemble 

•  Function  check 

Disassemble/assemble  breech  mechanism: 

•  Read  TM 

•  Disassemble 

•  Assemble 

•  Function  check 

Prepare  a  position  to  receive/emplace  a  howitzer: 

•  Install  stakes 

•  Establish  communication 

•  Record  information. 

The  analyses  were  conducted  on  these  sixteen  tasks  and  the 
following  six,  bringing  the  final  roster  to  22  tasks: 


•  Measure  quadrant  with  the  range  quadrant 

•  Set/lay  cannon  for  quadrant  with  range  quadrant 

•  Sight  on  a  target  with  the  direct  fire  telescope 

•  Perforin  gunner's  quadrant  micrometer  test 

•  Boresight  direct  fire  telescope  using  distant 
aiming  point  method 

•  Emplace/recover  aiming  posts. 

Note  that  the  first  step  of  Disassemble/assemble  the  breech 
mechanism  is  to  open  the  Technical  Manual  to  the 
appropriate  page;  this  was  scored  as  a  separate  task. 

Also,  note  that  Emplace/recover  aiming  posts  was  treated  as 
one  task,  since  the  two  components  do  not  involve  different 
task  characteristics. 


Data  Collection 

The  experiment  called  for  the  data  to  be  collected  at 
four  distinct  periods,  approximately  two  months  apart.  The 
first  period.  Acquisition,  was  conducted  June  13-17,  1983. 
All  soldiers  were  tested  and  trained  at  this  time.  The 
retention  tests  were  conducted  on  August  17-18,  November 
16-18,  and  January  16-19.  (These  tests  will  be  referred  to 
as  Retention  1,  Retention  2,  and  Retention  3,  respectively, 
throughout  the  remaining  text.  ) 

The  original  experimental  design  called  for 
participation  by  150  soldiers.  All  were  to  be  tested  and 
trained  during  Acquisition.  The  design  called  for  50 
soldiers  to  be  tested  at  all  three  retention  periods,  50 
more  to  be  tested  at  the  second  and  third  retention 
periods,  and  50  more  to  be  tested  at  the  third  retention 
period  only.  Table  4  shows  the  configuration  of  soldiers 
actually  tested  at  each  period.  Only  145  soldiers  were 
actually  tested  at  Acquisition. 
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Table  4:  Soldiers  Tested  During  Retention  Testing 


Subject 

Groups  Retention  1  Retention  2  Retention  3 


Three  Sessions 

60 

47 

51 

Two  Sessions 

43 

36 

One  Session 

27 

TOTAL 

60 

90 

114 

As  in  any  experiment  of  this  kind,  attrition  occurs. 
Of  the  original  145  soldiers  tested  during  Acquisition,  15 
were  lost  due  to  illness,  discharge,  or  transfer. 
Additionally,  some  soldiers  failed  to  take  tests  for  which 
they  were  scheduled.  The  resulting  test  groups  were: 

Tests  Taken  No.  of  Soldiers 


Retention 

1, 

2,  3 

41 

Retention 

2, 

3 

36 

Retention 

3 

27 

Retention 

1, 

2 

6 

Retention 

1, 

3 

10 

Retention 

1 

3 

Retention 

2 

7 

130 

The  last  four  groups  were  not  part  of  the  original  design; 
they  were  included  (or  excluded)  for  each  analysis  as 
appropriate . 

Scorers.  Scorers  for  the  data  collection  were  a 
combination  of  hired  civilians  and  NCOs  from  the  supporting 
battalion.  We  hired  seven  civilians  with  current  Field 
Artillery  experience  to  augment  scorers  available  in  the 
unit.  Six  of  the  civilians  were  senior  NCOS  or  officers  in 
a  National  Guard  battalion.  The  seventh  had  terminated 
active  duty  as  a  13a  within  six  months  of  testing.  The 
unit  provided  seven  NCOs  (E5  and  E6).  One  of  those  NCOs 
terminated  service  during  the  project  and  was  hired  to 
collect  data  on  the  remaining  phases  as  a  civilian. 
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The  sample.  The  soldiers  were  drawn  from  the  13B10 
MOS  at  Fort  Ord ,  California.  The  soldiers  carae  from  three 
batteries  and  ten  different  sections.  Forty-one  (28.3%)  of 
the  soldiers  listed  their  current  duty  position,  at  the 
time  of  Acquisition,  as  Cannoneer  Position  1.  The 
remaining  soldiers  listed  various  other  current  duty 
positions.  Most  of  the  sample  (61%)was  drawn  from  pay 
grade  E2.  The  OSUT  graduation  date  varied  between  four  and 
twelve  months  prior  to  Acquisition. 

There  was  nothing  in  the  selection  of  soldiers  for  the 
project  which  would  suggest  that  they  were  markedly 
different  from  the  average  U.S.  Army  entrant.  The  ASVAB 
aptitude  area  composite  scores  and  the  AFQT  scores  for  136 
of  the  soldiers  tested  at  Acquisition  are  shown  in  Table  5. 

We  compared  the  sample  ASVAB  means  to  a  sample  of 
210,162  soldiers  with  comparable  entry  dates.  While  the 
comparison  sample  included  all  MOSs,  we  found  few 
differences  between  the  two  groups.  Our  sample  had  lower 
mean  scores  (p  <.05)  on  the  Skilled  Technical  and  Clerical 
tests. 


Acquisition  Phase  Procedures 

During  the  Acquisition  phase,  soldiers  were  tested  on 
a  series  of  hands-on  tests  covering  the  selected  tasks.  If 
a  soldier  made  a  mistake  during  the  test,  the  scorer  told 
him  what  his  error  was  after  the  test  was  finished  and  had 
him  perform  the  task  again.  If  a  soldier  was  unable  to  do 
the  task  at  all,  the  scorer  walked  him  through  it.  During 
the  walk-through,  the  scorer  told  the  soldier  each  step  to 
perform  and  demonstrated  steps  if  necessary.  After  the 
walk-through,  the  soldier  performed  the  task  as  if  he  were 
being  tested  for  the  first  time.  This  process  was  repeated 
until  the  soldier  performed  the  task  without  error  or 
assistance. 

After  a  soldier  completed  this  initial  testing  and 
training  process  for  all  eleven  major  tasks,  ha  repeated 
the  process  two  more  times  for  about  half  the  tasks.  The 
clusters  of  tasks  receiving  additional  repetitions  are 
shown  in  Table  6.  (About  half  the  soldiers  repeated 
Cluster  A,  and  the  rest  repeated  Cluster  B.)  In  this 
report,  we  call  soldiers  who  completed  one  repetition  (one 
correct  performance)  on  a  task,  "proficient."  We  call 
soldiers  who  completed  three  repetitions,  "masters."  Thus, 
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Table  5 

Mean  ASVAB  Scores  for  Soldiers 
Participating  in  Acquisition  Test 
(N  =  136) 


Composite 

Mean 

Standard 

Deviation 

AFQT  Percentile  Score 

47.45 

17.97 

Combat 

100.24 

13.72 

Field  Artillery 

101.43 

10.36 

Electronics  Repair 

98.19 

11.49 

Operators/Food 

97.41 

13.53 

Surveillance/Communication 

98.20 

12.26 

Mechanical  Maintenance 

97.77 

13.21 

General  Maintenance 

97.64 

12.84 

Clerical 

98.21 

11.40 

Skilled  Technical 

96.93 

12.38 

General  Technical 

100.34 

13.38 
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Tafc  le  6 

Clusters  for  Additional  Acquisition  Trials 


A 


B 


Emplace/Recover  Collimator 

Emplace/Recover  Aiming  Posts 

Set/Lay  Cannon  for  Quadrant  with  the 
Range  Quadrant  (M198) 

Measure  the  Quadrant  with  the  Range 
Quadrant  (M198) 

Sight  on  a  Target  with  the  Direct 
Fire  Telescope  (M198) 

Disasserable/Assemble  Breech  and 
Firing  Mechanism  (Ml 98) 


Prepare  a  Position  to  Receive/ 
Emplace  a  Howitzer 

Perform  Operator  Maintenance  on 
the  M60  Machinegun 

Prepare  Separate  Loading  Ammunition 
for  Firing 

Boresight  the  Direct  Fire  Telescope 
Using  a  Distant  Aiming  Point  (DAP) 
(M198) 

Perform  Gunner's  Quadrant 
Micrometer  Test 
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all  soldiers  were  proficient  on  all  of  the  tasks  and 
masters  on  about  half  of  them. 

Within  a  repetition,  a  soldier  could  take  several 
trials  before  doing  the  task  without  error.  Scorers  rated 
the  first  trial  (after  the  walk-through,  if  there  was  one) 
in  detail.  They  rated  steps  performed  correctly  as  GO  and 
rated  steps  omitted  or  performed  incorrectly  as  NO  GO. 

They  also  kept  track  of  time  for  each  trial  (not  counting 
the  walk-through)  and  the  total  number  of  trials  (counting 
the  walk-through,  if  any)  for  each  repetition. 


Retention  Phase  Procedures 

At  Retention  1  and  Retention  2,  soldiers  performed 
each  task  once,  with  a  minimum  of  feedback  from  the 
scorers.  That  is,  scorers  did  not  give  any  information 
about  overall  performance  or  about  specific  steps.  Scorers 
rated  each  step  GO  or  NO  GO  and  recorded  the  time. 

Retention  3  was  similar  to  the  Acquisition  phase 
except  that  soldiers  did  only  one  repetition  (all  the 
trials  with  feedback  needed  to  perform  the  task  without 
error )  . 


Additional  Data  Collected 

At  each  phase,  soldiers  filled  out  a  form  which  asked 
questions  about  their  current  position  and  their  specific 
practice  history  for  the  tasks  tested.  During  the 
Acquisition  phase,  soldiers  reported  how  frequently  they 
had  performed  a  given  task  since  OSUT  and  how  recently  they 
had  performed  it.  During  the  Retention  phases,  soldiers 
answered  the  same  questions  but  only  for  the  time  period  of 
the  retention  interval.  The  questionnaires  are  shown  in 
Appendix  B. 

We  also  interviewed  the  Section  Chiefs  of  the  tested 
soldiers  after  each  phase.  At  that  time,  we  asked  them  to 
identify  each  soldier's  duty  position  and  to  indicate  the 
dates  for  their  major  training  events  (such  as  the  Army 
Readiness  Training  Evaluation  Program,  or  "ARTEP" ) .  We 
then  checked  the  Job  Books  for  the  soldiers.  The  Job  Book, 
if  kept  up  to  date,  indicates  the  last  time  that  the 
soldier  performed  the  task  under  the  Section  Chief's 
supervision.  The  forms  for  the  supervisor  are  shown  in 
Appendix  C. 


We  did  an  analysis,  where  data  were  available,  of  the 
self  reports  vs.  Section  Chief  and  Job  Book  data.  The 
results  of  this  analysis  can  be  found  in  Appendix  D. 


Acquisition  Results 

To  review  briefly,  during  Acquisition  testing  all 
(145)  soldiers  were  initially  tested  and  scored  on  all  22 
tasks  (six  primary  tasks  and  16  major  subtasks).  Task 
scoring  consisted  of  a  task-level  GO/NO  GO  measure  (i.e., 
whether  or  not  all  steps  were  performed  correctly), 
step-by-step  GO/NO  GO  measures,  and  (when  appropriate)  time 
for  task  performance.  Following  this  initial  testing,  all 
soldiers  were  trained  to  proficiency  on  all  tasks.  That 
is,  soldiers  were  given  feedback  on  their  performance, 
instructed  on  how  to  perform  the  task  correctly,  and 
retested  (and  rescored).  Thus,  we  were  able  to  collect  a 
rough  "acquisition  difficulty"  measure  —  namely.  Trials  to 
Proficiency:  how  many  times  the  soldier  had  to  be 

instructed  before  he  could  perform  the  task  correctly. 

After  performing  all  tasks  correctly,  each  soldier 
repeated  the  test-train-retest  cycle  twice  more  for  half  of 
the  tasks.  This  was  the  "Mastery"  training.  Performance 
was  scored  (GO/NO  GO,  steps  GO/NO  GO,  and  time)  for  each  of 
these  repetitions.  The  scores  from  the  three  repetitions 
of  the  Mastery  group  were  used  to  reflect  rate  of 
acquisition. 

First  Trial  Performance 

Performance  on  the  first  trial  of  Acquisition  varied 
markedly  across  tasks  (see  Table  7).  The  Advanced  tasks 
were  the  most  difficult.  Of  the  five  tasks  within  this 
group,  three  —  Boresight  Telescope,  Micrometer  Test,  and 
Sight  Target  —  were  passed  initially  by  fewer  than  25%  of 
the  soldiers.  Boresight  Telescope  was  the  most  difficult, 
as  assessed  by  both  accuracy  measures  (percentage  soldiers 
GO,  mean  steps  correct)  and  the  difficulty  measure  (trials 
to  proficiency).  Only  13.8%  of  the  soldiers  were  able  to 
complete  this  test  on  their  own;  the  others  were  talked 
through  the  test. 

Crew  tasks  enjoyed  the  highest  accuracy  rates.  No 
soldier  was  talked  through  any  of  these  tasks.  Emplace  and 
Recover  the  Collimator  and  Select  Powder  were  each  passed 
by  over  90%  of  the  soldiers  at  the  first  test. 


Table  7 

First-Trial  Acquisition  Performance 


Percentage  of  Trials  to  First 

Steps  Correct  First  Trial  Time  (Secs.)  Criterion 


Tasks 

Parcsntaga  of 
Soldiers  “GO" 

Mean 

Standard 

Deviation 

Mean 

Standard 

Deviation 

Mean 

Standard 

Deviation 

Measure  Quadrant 

46.9 

62.4 

47.3 

17.14 

8.90 

1.59 

.71 

Set/Lay  Cannon 

72.4 

82.2 

35.1 

21.20 

9.31 

1.29 

.48 

Sight  Target 

20.7 

31.7 

42.4 

16.84 

9.81 

1 .98 

.62 

Micrometer  Test 

24.3 

38.4 

46.6 

50.22 

16.99 

1,70 

.66 

Boresight  Telescope 

9.7 

12.0 

31.3 

140.49 

42.30 

2.31 

.80 

Emplace/Recover 

Aiming  Posts 

76.6 

96.4 

7.3 

79,63 

17.67 

1.21 

.42 

Emplace  Collimator 

94.4 

99.6 

1.9 

88.61 

19.00 

1.06 
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Recover  Collimator 

97.9 

87.2 

1.8 

41.62 

8.92 

1.02 

.14 

Select  Ammunition 

34.5 

83.1 

18.8 

Not  Timed 

1.69 

.57 

Select  Powder 

99.3 

99.7 

4.2 

Not  Timed 

1.69 

.57 

Set  Fuze 

83.4 

89.7 

24.9 

Not  Timed 

1.69 

.57 

Clear  M60  Machinegun 

89.6 

93.1 

23.2 

Not  Timed 

1.06 

.24 

Disassemble  M60 
Machinegun 

86.1 

93.4 

22.0 

134.64 

41.92 

1.17 

.41 

Assemble  M60  Machinegun 

93.0 

94.9 

21.7 

171.19 

41.75 

1.07 

.28 

Function  Check 

M60  Machinegun 

86.0 

92.2 

24.2 

Not  Timed 

1.15 

.43 

Read  TM-Breech 

21.4 

21.4 

0.4 

Not  Timed 

1.38 

.55 

Disassemble  Breech 

65.5 

55.2 

39.6 

72.45 

44.66 

1.38 

.55 

Assemble  Breech 

59.7 

65.2 

46.9 

123.92 

59.55 

1.41 

.52 

Function  Check  Breech 

65.5 

53.1 

37.7 

Not  Timed 

1.35 

.49 

install  Stakes 

27.6 

33.5 

45.6 

213.29 

82.36 

1.96 

.74 

Establish  Communication 

33.8 

48.0 

47.4 

213.20 

88.33 

1.81 

.72 

Record  Information 

39.3 

41.4 

48.4 

Not  Timed 

1.78 

.73 

Performance  cn  the  individual  tasks  varied  markedly. 
Tasks  associated  with  the  M60  machinegun  were  performed 
correctly  on  the  first  trial  by  over  85%  of  the  soldiers. 
Tasks  associated  with  preparing  a  position  (Install  Stakes, 
Establish  Communication,  Record  Information)  were  passed, 
on  the  average,  by  only  one-third  of  those  tested.  Read 
TM-Breech  was  the  only  individual  task  passed  by  fewer  than 
25%  of  the  soldiers. 

Previous  Experience 

Soldiers  were  asked  when  they  had  last  performed  any 
of  the  tasks  prior  to  the  Acquisition  training  and  testing. 
The  results  are  shown  in  Table  8.  (Please  note  that  only 
self-report  data  were  available.)  Ninety  percent  of  the 
participants  stated  they  had  not  performed  tasks  associated 
with  the  M60  machinegun  within  the  last  six  months.  In  the 
mcnth  preceding  testing,  the  Prepare  a  Position  tasks  had 
been  performed  by  only  16.6%  of  the  soldiers.  During  this 
same  time  frame,  Breech  tasks  had  been  performed  by 
approximately  40%  of  the  soldiers.  All  other  tasks  were 
reported  as  being  performed,  on  the  average,  by  a  third  of 
the  sample  the  month  before  Acquisition. 

The  self  reports  were  further  scrutinized  to 
investigate  the  reporting  pattern  of  individual  soldiers. 
Forty-nine  of  the  soldiers  (33.7%)  stated  that  they  had 
done  none  of  the  tasks  within  the  last  month.  Forty-five 
(31.0%)  stated  that  they  had  done  over  half  of  the  tasks  in 
the  last  month.  In  general,  the  soldiers  reported  doing  an 
average  of  four  tasks  within  the  last  month.  Nearly  all 
(97.9%)  of  the  soldiers  reported  doing  at  least  one  of  the 
tasks  in  the  last  six  months.  On  the  average,  eight  of  the 
tasks  were  done  by  soldiers  in  the  last  six  months. 

Mas tery /Prof iciency  Differences 

Tables  9-11  show  accuracy  measures  and  time  broken 
down  by  Mastery/Prof iciency  groups  and  by  the  three 
repetitions  of  the  Mastery  group.  We  looked  for 
differences  between  the  Mastery  and  Proficiency  groups  at 
the  first  trial  of  Acquisition.  We  found  few  statistically 
significant  differences  between  the  groups;  these 
differences  occurred  at  the  rate  that  we  might  have 
expected  if  left  to  chance.  Referring  to  Table  9,  only  one 
task  —  Set/Lay  Cannon  —  showed  significant  differences 
between  groups  on  first-trial  performance  (corrected 
Z  *  -1.80,  p  <  0.05).  Soldiers  in  the  proficiency  group 


Table  8 

Recency  Estimates  for  Task  Performance 
Prior  to  Acquisition  Test  (Self  Reports) 


%  Soldiers  %  Soldiers 

Reporting  Task  Reporting  Task 

Performance  Performance 

within  Last  Month  Between  1  and 

Tasks  6  Months  Ago 


Measure  Quadrant 

37.9 

34.5 

Set/Lav  Cannon 

30.3 

31.7 

Sight  Target 

31.7 

35.9 

Micrometer  Test 

37.2 

29.0 

Boresight  Telescope 

30.3 

30.3 

Emplace/Recover  Aiming  Posts 

30.3 

56.6 

Emplace  Collimator 

34.5 

52.4 

Recover  Collimator 

34.5 

52.4 

Select  Ammunition 

35.9 

45.5 

Select  Powder 

35.9 

45.5 

Set  Fuze 

35.9 

45.5 

Clear  M60  Machinegun 

5.0 

4.3 

Disassemble  M60  Machinegun 

5.0 

4.3 

Assemble  M60  Machinegun 

'3.0 

4.3 

Function  Check  M60  Machinegun 

5.0 

4.3 

Read  TM-Breech 

40.7 

37,9 

Disassemble  Breech 

40.7 

37.9 

Assemble  Breech 

39.3 

37.2 

Function  Check  Breech 

39.3 

372 

Install  Stakes 

16.6 

30.3 

Establish  Communication 

16.6 

30.3 

Record  Information 

16.6 

30.3 

%  Soldiers 
Reporting  No 
Task  Performance 
in  Last  6  Months 


27.6 
37.9 

32.4 
33.8 

39.3 

13.1 

13.1 

13.1 

18.6 

18.6 

18.6 

90.7 

90.7 

90.7 

90.7 

21.4 

21.4 

23.5 

23.5 

53.1 

53.1 

53.1 


Table  9 

Percentage  of  Soldiers  “GO”  on  Acquisition  Tests  by 

Training  Condition 


% 

$8 

a 

.*.'3 

<• 

TASKf 

1st 

Trial 

2nd  Trial 

3rd  Trial 

Proficiency 

N  1 

Mastery 

N  % 

% 

/ 

% 

Measure  Quadrant 

72 

47.2 

73 

46.6 

89.0 

95.8 

Set/Lay  Cannon 

72 

79.2* 

73 

65.8 

97.3 

98.6 

Sight  Target 

72 

25.0 

73 

16.4 

89.0 

94.3 

Micrometer  Test 

72 

25.0 

72 

23.6 

94.4 

98.6 

$ 

Foresight  Telescope 

71 

9.9 

73 

9.6 

91.7 

98.6 

Emplace/Recover 

,*& 

X 

Aiming  Posts 

72 

80.6 

69 

72.5 

98.6 

100.0 

m 

« 

Emplace  Collimator 

71 

95.8 

73 

93.2 

98.6 

98.5 

Recover  Collimator 

71 

98,6 

73 

97.3 

100.0 

98.5 

-  ..»;v 

Select  Ammunition 

70 

34.3 

75 

34.7 

93.1 

97.2 

Select  Powder 

70 

81.4 

75 

85.3 

100.0 

100.0 

& 

v 

•A 

,<V 

Set  Fuze 

70 

98.6 

75 

100.0 

98.6 

100.0 

1 

Clear  MGO  Machinegun 

72 

90.3 

72 

88.9 

100.0 

100.0 

Disassemble  M60 

■ ;» 

Machinegun 

72 

87.5 

72 

84.7 

98.6 

100.0 

Assemble  M60  Machinegun 

73 

91.8 

70 

94.3 

100.0 

100.0 

iy 

Function  Check  M60 

ft 

Machinegun 

73 

87.7 

70 

84.3 

100.0 

100.0 

,Q| 

Read  TM-Breech 

72 

23.6 

73 

19.2 

95.9 

100.0 

p 

& 

Disassemble  Ereech 

' 

72 

66.7 

73 

64.4 

94.3 

100.0 

jffl 

Assemble  Breech 

71 

63.4 

73 

56.2 

98.6 

100.0 

s 

Function  Check  Breech 

72 

66.7 

73 

64.4 

98.6 

98.6 

Install  Stakes 

78 

26.9 

67 

28.4 

91.0 

95.5 

t 

Establish  Communication 

78 

37.2 

6; 

29.9 

95.5 

97.0 

$ 

Record  Information 

78 

42.3 

67 

35.8 

97.0 

100.0 

$ 

| 

*  Difference  in  proportion 

significant  at 

the  ,05 

level. 
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Table  10 

Mean  Percentage  of  Steps  Correct  on  Acquisition  Tests 

by  Training  Condition 


1st 

Trial 

2nd  Trial 

3rd  Trial 

TASKS 

Proficiency 

N  Mean 

Mastery 

N  Mean 

Mean 

Mean 

Measure  Quadrant 

72 

64.0 

73 

61.0 

99.3 

100.0 

Set/Lay  Cannon 

72 

84.7 

73 

79.7 

99.7 

100.0 

Sight  Target 

72 

33.8 

73 

29.  7 

97.7 

97.6 

Micrometer  Test 

72 

39.4 

72 

37.5 

97.9 

100.0 

Boresight  Telescope 

71 

14.1 

73 

9.9 

99.0 

100.0 

Emplace/Recover 

Aiming  Posts 

72 

97.1 

69 

95.7 

99.8 

100.0 

Emplace  Collimator 

71 

99,7 

73 

99.4 

99.9 

99.9 

Recover  Collimator 

71 

99.7 

73 

99.8 

100.0 

99.9 

Select  Ammunition 

70 

84.0 

75 

82.4 

99.0 

99.6 

Select  Powder 

70 

99.3 

75 

100.0 

100.0 

100.0 

Set  Fuze 

70 

90.0 

75 

89.3 

99.3 

100.0 

Clear  M60  Machinegun 

72 

93.8 

72 

92.4 

100.0 

100.0 

Disassemble  M60 

Machinegun 

72 

94.0 

72 

92.8 

99.8 

100.0 

Assemble  M60  Machinegun 

73 

95.5 

70 

94.3 

100.0 

100.0 

Function  Check  M60 
Machinegun 

73 

92.9 

70 

91.4 

100.0 

100.0 

Read  TM-Breech 

72 

23.6 

73 

19.2 

95.9 

100.0 

Disassemble  Breech 

72 

68.1 

73 

64.4 

98.3 

100.0 

Assemble  Breech 

72 

67.1 

73 

63.4 

74.8 

100.0 

Function  Check  Breech 

72 

67.4 

73 

64.4 

99.3 

99.3 

Install  Stakes 

78 

34.0 

67 

32.8 

97.4 

98.9 

Establish  Coi  iraunication 

78 

51.0 

67 

44.4 

99.5 

99.7 

Record  Information 

78 

44.2 

67 

38.1 

98.5 

100.0 
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Table  11 

Mean  Time  to  Perform  Tasks  on  Acquisition  Trials 
by  Training  Condition 


Tasks 

Proficiency 

Mastery 

N 

1st  Trial 
Mean 

Pinal  Trial 
Mean 

N 

1st  Trial 
Mean 

2nd  Trial 
Mean 

3rd  Trial 

Mean 

Measure  Quadrant 

72 

17.3 

13.2 

73 

17.0 

14.3 

13.2 

Set/Lay  Cannon 

69 

21.3 

19.8 

73 

21.1 

16.8 

16.9 

Sight  Target 

70 

15.5 

10.9 

73 

18.1 

11.4 

8.9 

Micrometer  Test 

71 

51.0 

44.5 

72 

49.5 

41.4 

38.4 

Boresight  Telescope 

69 

139.6 

130.6 

73 

141  4 

128.7 

109.3 

Empiace/Recover 

Aiming  Posts 

72 

78.4 

79.4 

69 

80.9 

74.8 

69.9 

Emplace  Collimator 

71 

89.1 

88.0 

73 

88.1 

75.0 

71.7 

Recover  Collimator 

69 

42.4 

42.4 

73 

40.9 

37.0 

36.9 

Select  Ammunition 

Not  Timed 

Select  Powder 

Not  Timed 

Set  Fuze 

Not  Timed 

Clear  M60  Machinegun 

Not  Timed 

Disassemble  M60 

Machinegun 

72 

130.1 

131.2 

72 

139.2 

120.8 

115.1 

Assemble  M60  Machinegun 

73 

171.3 

168.3 

70 

171.0 

159.4 

153.7 

Function  Check 

M60  Machinegun 

Read  TM-Breech 

Not  Timed 

Disassemble  Breech 

72 

71.0 

70.9 

73 

73.9 

55.0 

46.2 

Assemble  Breech 

70 

113.5* 

112.5 

73 

133.9 

103.9 

92.3 

Function  Check  Breech 

Not  Timed 

Install  Stakes 

75 

216.8 

208.3 

S7 

209.3 

167  8 

142.2 

Establish  Communication 

74 

232.0 

226,5 

67 

230.3 

171  1 

153.0 

Record  Information 

Not  Timed 

*  Difference  of  means  significant  at  05  level. 
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performed  the  proficiency  group  scored  "GO"  on  this  test. 

No  differences  were  found  between  groups  on  mean  percentage 
of  steps  correct  (Table  10).  For  the  time  variable 
(Table  11),  Assemble  Breech  was  the  only  task  which  showed 
a  statistically  significant  difference  between  groups 
(t  =  -2.07,  p  <  0.05),  the  proficient  soldiers  performing 
the  task  faster.  These  initial  differences  between 
Proficiency  and  Mastery  groups  were  controlled  for  in 
subsequent  analyses. 

Rate  of  Acquisition 

The  learning  rates  are  indicated  in  several  ways.  A 
first  indicator  is  the  number  of  trials  required  to 
successfully  reach  proficiency.  Table  7  shows  these  means. 
This  indicator  parallels  the  level  of  difficulty  indicated 
by  the  percent  soldiers  "GO,"  with  Boresight  Telescope 
having  the  highest  mean  number  of  trials,  and  Recover 
Collimator  the  least.  Boresight  Telescope  was  the  only 
task  for  which  the  mean  exceeded  two  trials.  Second,  the 
learning  rate  can  be  indicated  by  the  performance  across 
trials  of  the  Mastery  group.  The  two  accuracy  measures, 
shown  in  Tables  9  and  10,  indicate  that  soldiers  learned 
the  tasks  quickly.  Even  the  most  difficult  tasks  at  the 
first  trial  of  Acquisition  —  Boresight  Telescope, 

Micr  meter  Test,  Sight  Target,  and  Read  TM-Breech  —  were 
passed  by  at  least  89%  of  the  Mastery  soldiers  the  second 
time  around.  Eleven  of  the  tasks  were  passed  by  100%  of 
the  soldiers  by  the  third  trial;  fifteen  of  the  tasks  held 
all  steps  passed  by  all  soldiers  in  this  third  trial,  time 
being  the  "NO  GO"  determinant  for  four  of  these  15. 

Time  provides  a  third  indicator.  Table  11  shows  the 
average  time  required  to  perform  the  timed  tasks. 

Virtually  every  succeeding  trial  showed  a  decrease  in  the 
amount  of  time  the  soldier  required  to  complete  the  task. 
The  differences  between  the  first  and  second  trials  were 
the  most  dramatic.  On  the  average  the  time  decreased  by 
18%  between  the  two  trials.  Time  was  reduced  by  an  average 
of  9%  across  all  tasks  between  the  second  and  third  trials. 
Acquisition  is  also  reflected  by  the  improvements  in  time 
between  the  first  trial  (before  training)  and  last  trial 
(after  training)  for  the  Proficiency  group. 
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Summary  of  Acquisition  Performance 

The  accuracy  scores  for  the  first  trial  of  acquisition 
show  that  the  soldiers  included  in  the  experiment  had 
differing  levels  of  proficiency  on  the  selected  tasks.  The 
soldiers  themselves  reported  a  wide  range  of  experience  on 
the  tasks  in  the  six  months  prior  to  the  test.  Performance 
on  all  tasks  improved  substantially  with  training: 
accuracy  approached  100%  for  the  Mastery  group  by  the  third 
repetition.  Performance  time  continued  to  decrease  during 
the  third  repetition.  These  findings  parallel  our 
experience  with  the  11B  tasks  examined  during  Year  II. 

As  a  consequence  of  the  training  provided  during 
Acquisition,  performance  time  baselines  were  established 
against  which  to  monitor  proficiency  levels  during 
Retention  testing.  For  the  proficiency  group  these 
baselines  were  the  soldiers'  times  on  their  final  (correct) 
Acquisition  test  trial.  For  the  Mastery  group  the  baseline 
was  the  time  on  their  third  correct  trial. 

Retention  Results 

The  retention  tests  were  given  two,  five,  and  seven 
months  after  the  Acquisition  phase.  Of  central  interest  is 
the  effect  of  these  time  intervals  on  performance  levels. 
Also  of  interest  are  differences  in  retention  performance 
attributable  to  training  effects  —  the  Proficiency  vs. 
Mastery  regimens.  We  will  first  present  the  descriptive 
results  from  each  testing  session  and  introduce  the 
variables  apparently  influencing  performance.  This 
discussion  will  be  followed  by  a  more  detailed  presentation 
of  analyses  aimed  at  predicting  performance  across 
retention  intervals  (see  Chapter  IV). 

Retention  Performance:  Accuracy 

As  a  global  overview,  we  will  first  present 
performance  summaries  for  all  soldiers  tested  at  each 
retention  point.  These  summaries  combine  soldiers  from 
different  subgroups  (e.g..  Mastery  and  Proficiency  groups, 
soldiers  previously  tested  at  Retention  1  or  2  and  those 
not  previously  tested,  etc.);  breakouts  of  these  subgroups 
will  be  presented  where  appropriate. 

Retention  performance  for  the  accuracy  measures  is 
shown  in  Tables  12  (Percentage  of  Soldiers  "GO")  and  13 
(Percentage  of  Steps  "GO")  for  all  soldiers  tested.  It  is 


Table  12 

Retention  Performance  All  Soldiers: 
Percentage  of  Soldiers  “GO” 


Tasks 

Retention  1 
(n  =  60) 

Retention  2 
(n  »  90) 

Retention  3 

(n  *  114) 

Measure  Quadrant 

46.7 

61.1 

72.8 

Set/Lay  Cannon 

58.3 

50.0 

69.3 

Sight  Target 

35.0 

17.8 

36.0 

Micrometer  Test 

31.7 

44.4 

70.8 

3oresight  Telescope 

19.0 

20.2 

45.6 

Emplace/Recover  Aiming  Posts 

70.0 

52.2 

96.5 

Emplace  Collimator 

95.0 

48.9 

93  0 

Recover  Collimator 

96.7 

56.7 

91.2 

Select  Ammunition 

63.3 

9.1 

29.8 

Select  Powder 

100.0 

100.0 

99.1 

Set  Fuze 

100.0 

65.1 

* 

Clear  M60  Machinegun 

90.0 

85.6 

99.1 

Disassemble  M60  Machinegun 

80.0 

44.9 

92.0 

Assemble  M60  Machinegun 

83.3 

82.2 

87  7 

Function  Check  M60  Machinegun 

98.3 

95.6 

100.0 

Read  TM-Breech 

88.3 

72.2 

78.1 

Disassemble  Breech 

90.0 

92.2 

95.6 

Assemble  Breech 

93.3 

70.0 

84.2 

Function  Check  Breech 

95.0 

71.1 

79.8 

install  Stakes 

76.7 

61.1 

86.0 

Establish  Communication 

76.7 

62.2 

84.2 

Record  Information 

52.6 

57  8 

57.0 

*  Fuze  not  available  during  testing 
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Table  13 

Retention  Performance  All  Soldiers: 
Percentage  of  Steps  “GO” 


Retention  1 
(n  *  60) 

Retention  2 
(n  =  90) 

Retention  3 
(n  =  114) 

Tasks 

Mean 

Standard 

Deviation 

Mean 

Standard 

Deviation 

Mean 

Standai 

Deviatk 

Measure  Quadrant 

70.3 

41.6 

76.7 

41.2 

63.3 

31.7 

Set/Lay  Cannon 

90.8 

17.1 

96.7 

16.1 

90.5 

17.1 

Sight  Target 

72.3 

29.8 

62.2 

27.0 

65.5 

31.3 

Micrometer  Test 

73.8 

35.0 

80.1 

32.1 

91.8 

17  8 

Soresight  Telescope 

43.8 

41.4 

52.4 

37.8 

75.5 

34.6 

Emplace/Recover 

Aiming  Posts 

96.4 

7.1 

87.1 

14.9 

88.0 

4.2 

Emplace  Collimator 

99.7 

1.4 

95  3 

6.5 

99.5 

2.1 

Recover  Collimator 

99.8 

1.8 

94.4 

7.0 

98.7 

4.1 

Select  Ammunition 

90.0 

17.3 

68.6 

21.0 

87.1 

11.7 

Select  Powder 

100.0 

0.0 

100  0 

0.0 

99.6 

4.7 

Set  Fuze 

100.0 

0.0 

79.5 

30.3 

* 

* 

Clear  M60  Machinegun 

96.3 

12.9 

87.8 

30.7 

99.8 

2  4 

Disassemble  M60 

Machinegun 

96.1 

7.7 

88.0 

16.3 

98.1 

7  3 

Assemble  M60  Machinegun 

99.0 

3.6 

97.8 

9.3 

99.0 

3.7 

Function  Check 

M60  Machinegun 

99.7 

2.6 

96.4 

18.1 

100.0 

0.0 

Read  TM-Breech 

88.3 

32.4 

72.2 

45.0 

78.1 

41.6 

Disassemble  Breech 

95.3 

17.4 

97.1 

12.9 

98.4 

10.0 

Assemble  Breech 

98.5 

6.9 

94.4 

12.2 

96.2 

9.3 

Function  Check  Breech 

96,7 

15.6 

81.7 

31.3 

89.5 

21  5 

Install  Stakes 

94  2 

10.7 

88.9 

15.0 

94  1 

18.3 

Establish  Communication 

96.5 

6.9 

94.2 

9.3 

96.8 

10.6 

Record  information 

72.8 

31.4 

75  0 

32  0 

74  6 

32.1 

'  Fuze  not  available  for  testing 
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evident  from  these  tables  that  practically  all  tasks  showed 
forgetting  after  two  months:  since  (by  design)  all 
soldiers  performed  all  tasks  correctly  on  their  last 
Acquisition  test,  any  accuracy  score  less  than  100%  is  an 
indication  of  retention  loss.  The  task  groups  —  Advanced, 
Crew,  and  Individual  —  maintained  their  relative  positions 
regarding  level  of  difficulty  throughout  the  three 
Retention  tests.  That  is,  at  each  Retention  test,  the 
Advanced  tasks  were  the  most  difficult,  followed  by  Crew 
tasks,  then  Individual  tasks.  The  Advanced  tasks, 
characterized  as  the  most  difficult  at  Acquisition,  showed 
the  lowest  retention.  Individual  tasks  showed  the  highest 
rates  of  retention.  On  the  average,  84%  of  the  soldiers 
passed  these  tasks  at  the  end  of  the  first  Retention 
interval . 

At  Retention  2,  all  but  one  of  the  tasks  showed 
forgetting  since  Acquisition.  Thj  most  dramatic  drop  was 
in  Select  Ammunition;  only  9.1%  of  the  soldiers  passed 
this  test,  although  68.6%  of  the  steps  were  performed 
correctly.  Emplace  Collimator,  Set  Fuze,  and  Disassemble 
M60  Machinegun  each  showed  retention  losses  of  over  35% 
from  the  first  to  the  second  retention  testing.  Select 
Powder  continued  to  be  passed  by  all  soldiers  tested. 

The  performance  of  soldiers  at  Retention  3  is  more 
difficult  to  explain.  On  virtually  every  task  soldiers 
performed  better  or  as  well  at  this  testing  than  at  the 
previous  one.  Comparing  performance  across  tests,  we  find 
that  at  both  Retention  1  and  3  fourteen  of  the  22  tasks 
were  passed  by  75%  or  more  of  the  soldiers.  At 
Retention  2,  however,  only  five  of  the  tasks  fell  into  this 
category.  All  tasks  were  passed  by  at  least  25%  of  the 
soldiers  at  Retention  3.  At  Retention  2,  three  tasks  — 
Select  Ammunition,  Sight  Target,  and  Boresight  Telescope  — 
were  passed  by  less  than  25%. 

There  are  several  potential  confounding  factors, 
including  testing  and  practice  effects,  which  serve  to 
explain  the  Retention  3  results.  This  topic  is  taken  up  in 
more  detail  in  Chapter  IV. 

Retention  Performance:  Time 

Table  14  presents  Retention  results  for  task 
performance  time,  collapsed  across  all  soldiers  at  each 
retention  test.  These  data  are  included  for  the  sake  of 
completeness?  however,  they  should  be  viewed  cautiously. 


40 


■egragCTwngcnBroggHcr 


Table  14 

Retention  Performance  for  Timed  Tasks  All  Soldiers: 
Performance  Time  (Seconds) 


TASKS 

Retention  1 

Retention  2 

Retention  3 

Mean 

Standard 

Deviation 

Standard 
Mean  Deviation 

Standard 
Mean  Deviation 

Measure  Quadrant 

17.8 

13.1 

15.5 

10.6 

14.5 

6.2 

Set /Lay  Cannon 

24.0 

11.1 

18.  / 

7.2 

23.3 

8.7 

Sight  Target 

16.5 

11.4 

16.2 

8.6 

14.5 

7.5 

Micrometer  Test 

83.4 

53.6 

80.3 

47.6 

51.1 

16.0 

Boresight  Telescope 

147.7 

61.7 

120.7 

54.6 

118.1 

38.9 

Emplace/Recover 

Aiming  Posts 

80.4 

14.9 

74.5 

12.3 

78.2 

13.0 

Emplace  Collimator 

84.4 

12.6 

95.2 

23.1 

r  3 .  o 

19.9 

Recover  Collimator 

37.9 

6.1 

43.4 

13.5 

44.7 

9.7 

Disassemble  M60  Machinegun 

151.3 

39.7 

176.9 

50.4 

149.9 

41.0 

Assemble  M60  Machinegun 

198.7 

58.1 

198.5 

44.1 

175.3 

43.6 

Disassemble  Breech 

97.0 

54.8 

104.5 

68.3 

87.4 

43.1 

Assemble  Breech 

150.3 

107.5 

151.6 

95.9 

130.1 

74.5 

Install  Stakes 

111.6 

34.7 

112.9 

41.6 

106.3 

36.4 

Establish  Communication 

140.8 

32.6 

148.4 

41.6 

151.6 

40.7 
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For  example,  times  for  task  performance  at  Retention  2 
include  soldiers  tested  at  Retention  1;  therefore,  direct 
comparisons  between  these  means  would  be  potentially 
misleading. 

To  more  accurately  describe  changes  in  speed  of 
performance  between  Acquisition  and  Retention  performance, 
we  compared  the  means  between  the  last  Acquisition  trial 
time  (regardless  of  training  condition)  and  the  first  trial 
at  each  Retention  session.  The  results  are  shown  in 
Table  15.  Thus,  for  example,  the  means  listed  under 
"Retention  2"  are  those  of  just  the  soldiers  whose  first 
retention  test  was  at  Retention  2.  Calculating  these  means 
enabled  us  to  statistically  test  for  significant 
differences  between  Acquisition  and  Retention  performance. 

For  the  majority  of  tasks,  more  time  was  required  for 
task  performance  during  Retention  tests  than  at  the  end  of 
Acquisition.  In  other  words,  the  ability  to  perform  the 
tasks  diminished  during  the  retention  intervals.  Two  major 
exceptions  are  apparent.  Regardless  of  the  time  of  a 
soldier's  first  retention  test,  the  times  required  to 
Install  Stakes  and  Establish  Communication  were 
significantly  reduced  compared  to  the  last  Acquisition 
trial.  This  time  decrease  was  not  what  would  have  been 
expected  if  the  major  effect  of  the  retention  interval  was 
to  slow  performance. 

Three  tasks  —  Sight  Target,  Disassemble  Breech,  and 
Assemble  Breech  —  show  relatively  large  (greater  than  50%) 
and  consistent  increases  in  performance  time  between 
Acquisition  and  Retention  for  all  retention  sessions.  At 
Retention  1,  six  of  the  tasks  for  which  significant 
differences  were  found  showed  increases  in  performance  time 
of  less  than  25%.  Somewhat  surprisingly,  mere  tasks  (n=10) 
showed  increases  in  performance  time  at  Retention  1  than  at 
either  Retention  2  (n®7)  or  Retention  3  (n*8). 

Furthermore,  Retention  1  increases  tended  to  be  larger. 

We  also  analyzed  performance  times  at  Retention  2  and 
3,  comparing  soldiers  previously  tested  with  those  who  were 
taking  their  first  retention  test.  At  Retention  2,  no 
significant  differences  were  found  between  the  two  groups 
on  any  task.  At  Retention  3,  significant  differences  were 
found  between  groups  only  on  Install  Stakes  and  Set/Lay 
Cannon.  In  each  instance,  the  untested  group  took  more 
time  to  complete  the  task  than  those  in  the  group  which  had 
been  previously  tested. 
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Table  15 

First  Retention  Performances  Compared  to  Last  Trial  Acquisition 

Mean  Time  (Seconds) 


Retention  1  Retention  2  Retention  3 


(N«60)a 

(N=43)a 

(N=27) 

a 

Mean 

Acquisition 

Mean 

Retention  1 

Mean 

Acquisition 

Mean 

Retention  2 

Mean  Mean 

Acquisition  Retention  3 

TASKS 

Measure  Quadrant 

13.0* 

17.8 

13.3 

16.3 

13.2 

13.7 

Set/Lay  Cannon 

18.2* 

23.6 

18.7 

18.9 

18.7 

21.1 

Sight  Target 

9.7* 

15.6 

10.0* 

16.2 

10.3* 

16.5 

Micrometer  Test 

40.4* 

82.8 

44.5* 

83.5 

41.2* 

54.3 

Boresight  Telescope 

118.5* 

148.4 

CM 
f— i 

118.0 

116.1 

113.0 

Emplace/Recover 

Aiming  Posts 

72.4* 

80. C 

75.0 

73.2 

80.0 

80.9 

Emplace  Collimator 

81.0 

84.6 

79.2* 

94.1 

78.7* 

87.4 

Recover  Collimator 

39.5 

38.0 

41.4 

46.2 

37.2* 

42.8 

Disassemble  M60 
Machinegun 

122.5* 

151.3 

123.4* 

184.6 

122.2* 

149.3 

Assemble  M60  Machinegun 

161.4* 

198.0 

161.5* 

200.0 

157.2* 

182.8 

Disassemble  Breech 

54.6* 

97.4 

63.4* 

108.1 

58.9* 

94.6 

Assemble  Breech 

102.1* 

150.  6 

94.4* 

150.6 

107.0* 

160.7 

Install  Stakes 

180.5* 

109.7 

164.2* 

114.8 

182.8* 

130.2 

Establish  Communication 

198.4* 

140.1 

172.2 

153.4 

194.0* 

156.7 

*  Difference  between  means  significant  at  .05  level,  usine  2-tailed  t-tests. 


These  N's  are  approximate  and  vary  among  tasks  due  to  missing  data 
for  the  last  trial  Acquisiton  times - 


fable  16 

Retention  Performance  ail  Soldiers:  Percentage  of 
Soldiers  “GO”  by  Training  Condition 


Retention  1 _  Retention  2  Retention  3 


TASKS 

% 

(N-60) 

Standard 

Deviation 

7. 

(N=90) 

Standard 

Deviation 

% 

(N=114) 

Standard 

Deviation 

Measure  Quadrant 

P 

42.3 

50.4 

68.2 

47.1 

74.5 

44.0 

M 

50.0 

50.8 

54.3 

50.4 

71.2 

45.7 

Set/Lay  Cannon 

P 

46.2* 

50.8 

56.8 

50.1 

72.7 

44.9 

M 

67.  7 

47.5 

43.5 

50.1 

66.1 

47.7 

Sight  Target 

P 

30.8* 

47.1 

20.5 

40.8 

41.8 

49.8 

M 

38.2 

49.3 

15.2 

36.3 

30.5 

46.4 

Micrometer  Test 

P 

27.3 

45.2 

36.2 

48.6 

76.3 

42.9 

M 

37.0 

49.2 

53.5 

50.5 

64.8 

48.2 

Boresight  Telescope 

P 

22.6 

42.5 

20.0 

40.5 

42.4 

49.8 

M 

14.8 

36.2 

20.5 

40.8 

49.1 

50.5 

Emplace/Recover 

M 

80.8 

40.2 

47.7 

50.5 

96.4 

18.9 

Aiming  Posts 

P 

67.8 

47.5 

59.1 

49.7 

96.5 

18.6 

Emplace  Collimator 

P 

96.2 

19.6 

51.2 

50.6 

90.7 

29.3 

M 

94.1 

23.9 

45.7 

50.4 

94.9 

22.2 

Recover  Collimator 

P 

100.0 

0 

55.8 

50.2 

90.7 

29.3 

M 

94.1 

23.9 

56.5 

50.1 

91.5 

28.1 

Select  Ammunition 

P 

64.5 

48.6 

9.5 

29.7 

23.2 

42.6 

M 

62.1 

49.4 

8.7 

28.5 

36.2 

48.5 

Select  Powder 

P 

100.0 

0 

100.0 

0 

100.0 

0 

M 

100.0 

0 

100.0 

0 

98.3 

13.1 

Set  Fuse 

P 

100.0 

0 

61.0 

49.4 

** 

M 

100.0 

0 

69.0 

46.8 

** 

Clear  M60  Machinegun 

P 

85.3 

35.9 

86.7 

34.4 

98.3 

13.1 

M 

96.2 

19.6 

84.1 

37.0 

100.0 

0 

Disassemble  M60 

P 

76.5 

43.1 

43.2 

50.1 

93.1 

25.6 

Machinegun 

M 

84.6 

36.8 

45.5 

50.4 

92.6 

26.4 
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Table  16  (continued) 


TASKS 

Retention  1 

Retention  2 

Retention  3 

% 

(N=60) 

Standard 

Deviation 

1 

(N=90) 

Standard 

Deviation 

Z 

(N-114) 

Standard 

Deviatioi 

Assemble  M60  Machinegun 

P 

73.5* ** 

44.8 

34.8 

36.3 

86.4 

-4-5 

M 

96.2 

19.6 

81.0 

39.7 

90.6 

29.5 

Function  Check 

P 

97.1 

17.1 

93.5 

25.0 

100.0 

0 

M60  Machinegun 

M 

100.0 

0 

97.6 

15.4 

100.0 

0 

Read  TM-Breech 

P 

80.8 

40.2 

77.3 

42.4 

78.2 

41.7 

M 

94.1 

23.9 

67.4 

47.4 

78.0 

41.8 

Diassemble  Breech 

P 

88.5 

32.6 

93.2 

25.5 

94.5 

22.9 

M 

91.2 

28.8 

91.3 

28.5 

96.6 

18.3 

Assemble  Breech 

P 

96.2 

19.6 

79.5* 

40.8 

83.6 

37.3 

M 

91.8 

28.8 

60.9 

49.3 

84.7 

36.  ^ 

Function  Check  Breech 

P 

92.3 

27.2 

63.6 

43.7 

78.2 

41.7 

M 

97.1 

17.1 

78.3 

41.7 

81.4 

39.3 

Install  Stakes 

P 

80.0 

40.6 

55.1 

50.3 

83.6 

37.3 

M 

72.0 

45.8 

68.3 

47.1 

88.7 

32.0 

Establish  Communication 

P 

74.3 

44.3 

63.3 

48,  7 

80.3 

40.1 

M 

80.0 

40.8 

61.0 

49.4 

38.7 

32.0 

Record  Information 

P 

31.3* 

47.1 

55.1 

50.3 

42.6* 

49.9 

M 

80.0 

40.  8 

61.0 

49.4 

73.6 

44.5 

*  Difference  between  proportions  significant  at  .05  .level. 

**  Fuze  not  available  for  testing. 

Note:  P«=Prof  iciency ;  M«*Mastery 
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Table  17 

Retention  Performance  all  Soldiers:  Mean  Percentage  of 
Steps  Correct  by  Training  Condition 


Retention  1 

Retention  2 

Retention  3 

(N=60) 

(N=90) 

(N*=114) 

Standard 
%  Deviation 

Standard 
%  Deviation 

Standard 
%  Deviation 

TASKS 

Measure  Quadrant 

P 

66.0 

45.0 

78.4 

39.5 

83.6 

30.5 

M 

73.5 

39.4 

75.0 

43.1 

83.1 

33.0 

Set/Lay  Cannon 

P 

91.2 

18.3 

88.2 

15.1 

90.9 

17.1 

M 

90.6 

16.5 

85.2 

17.1 

90.2 

17.2 

Sight  Target 

P 

74.7 

30.9 

59.1 

29.5 

69.7 

30.9 

M 

70.6 

29.3 

65.2 

24.3 

61.6 

31.4 

Micrometer  Test 

P 

74.6 

37.7 

76.6 

34.2 

92.8 

18.4 

M 

72.7 

32.0 

84.0 

29.5 

90.7 

17.2 

Boresight  Telescope 

P 

44.0 

44.1 

49.7 

39.4 

76.5 

33.2 

M 

43.5 

38.8 

55.1 

36.3 

74.5 

36.2 

Emp.l ace/Recover 

M 

97.5 

6.3 

86.4 

14.7 

88.4 

5.3 

Aiming  Posts 

P 

95.7 

7.7 

88.4 

15.3 

87.7 

2.9 

Emplace  Collimator 

P 

99.8 

1.2 

91.5 

6.6 

99.4 

2.2 

M 

99.6 

1.2 

9  J. 0 

6.5 

99.6 

2.0 

Recover  Collimator 

P 

10C.0 

0 

£4.3 

7.1 

98.7 

4.2 

M 

99.6 

2.5 

M.4 

7.0 

98.8 

4.0 

Select  Ammunition 

P 

90.0 

18.8 

66.4 

22.5 

85.2 

12.6 

M 

90.0 

15.8 

70.7 

19.6 

89.0 

10.5 

Select  Powder 

P 

100.0 

0 

10C.0 

0 

100.0 

0 

M 

100.0 

0 

100.0 

0 

99.1 

6.6 

Set  Fuze 

P 

100.0 

0 

76.8 

31.8 

** 

M 

100.0 

0 

82.1 

28.8 

** 

Clear  M60  Machinegun 

P 

94.1 

16.4 

83.3 

30.4 

99  6 

3.3 

M 

99.0 

4.9 

86.9 

31.7 

10C.0 

0 

Disassemble  M60 

P 

95.1 

8.7 

86.0 

19.9 

58.6 

6.5 

Machinegun 

M 

97.4 

6.1 

89.8 

11.5 

97.8 

8.0 
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Table  17  (continued) 


Retention  1 

Retention  2 

Retention  3 

(N®60) 

(N=90) 

(N«114) 

Standard 

Standard 

Standard 

%  Deviation 

X  Deviation 

X  Deviation 

TASKS 


Assemble  M60  Machinegun 

P 

98.3 

4.  7 

98.3 

12.2 

98.5 

4.4 

M 

100.0 

0 

100.0 

4.  7 

99.5 

2.7 

Function  Check 

P 

99.4 

3.4 

95.2 

20.  7 

100.0 

0 

M6C  Machinegun 

M 

10C.0 

0 

97.6 

15.4 

100.0 

0 

Read  TM-Breech 

P 

80.8 

40.2 

77.3 

42.4 

78.2 

41.8 

M 

94.1 

23.9 

67.4 

47.4 

78.0 

41.8 

Diassemble  Breech 

P 

94.  6 

20.0 

98.6 

5.1 

97.4 

13.9 

M 

95.9 

15.4 

95.  7 

17.3 

99.3 

3.7 

Assemble  Breach 

P 

100.0 

0 

97.2* 

7.1 

95.7 

10.3 

M 

97.4 

9.1 

91.8 

15.2 

96.6 

8.3 

Function  Check  Breech 

P 

94.2 

21.6 

76.1 

34.9 

89.1 

20.8 

M 

98.5 

8.6 

86.9 

26.7 

89.8 

22.3 

Install  Stakes 

P 

95.0 

10.1 

86.7 

16.2 

91.8 

23.1 

M 

93.0 

11.5 

91.4 

13.2 

96.7 

9.8 

Establish  Communication 

? 

96.2 

7.3 

94.8 

7.9 

95.8 

13.6 

M 

96.9 

6.8 

93.5 

10.8 

97.9 

5.3 

Record  Information 

P 

59.4* 

32.2 

70.4 

36.7 

65.5* 

33.6 

M 

90.0 

20.4 

80.5 

24.7 

84.9 

27.0 

*  Difference  between  proportions  significant  at  .05  level, 
**  Fuze  not  available  for  testing. 
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Table  18 

Retention  Performance  for  Timed  Tasks  All  Soldiers 
Performance  Time  (Seconds)  by  Training  Condition 


Tasks 

Retention  1 
(n  =  60) 

Retention  2 
(n  *  90) 

Retention  3 
(n  *  114) 

Mean 

Standard 

Deviation 

Mean 

Standard 

Deviation 

Mean 

Standard 

Deviation 

P 

18.1 

13.2 

14.9 

7.9 

14.0 

6.5 

Measure  Quadrant 

M 

17.5 

13.2 

16.1 

13.0 

15.0 

6.0 

P 

27.6’ 

13.3 

18.6 

8.0 

22.8 

6.5 

Set/Lay  Cannon 

M 

21.4 

8.4 

18.9 

6.4 

23.7 

10.5 

P 

19.3* 

13.6 

15.8 

7.5 

14.9 

7.6 

Sight  Target 

M 

14.4 

9.0 

16.6 

9.6 

14.0 

7.4 

P 

39.8 

59.7 

86.9* 

50.2 

50.4 

15.5 

Micrometer  Test 

M 

75.8 

45.3 

73.0 

44.0 

51.7 

16.5 

P 

154.3 

63.7 

129.5 

57.4 

120.5 

39.5 

Boresight  Telescope 

M 

139.5 

59.5 

111.7 

50.4 

116.7 

38.6 

Emplace/Recover 

P 

81.1 

15.8 

72.6 

11.8 

78.7 

13.8 

Aiming  Posts 

M 

79.7 

14.6 

76.4 

12.2 

77.8 

12.4 

P 

85.1 

13.2 

96.9 

22.7 

95.2 

22.6 

Emplace  Collimator 

M 

83.9 

12.4 

93.5 

23.8 

90.8 

17.1 

P 

36.6 

5.6 

41.6 

9.7 

44.9 

11.6 

Recover  Collimator 

M 

38.9 

6.5 

45.2 

16.3 

44.4 

7.6 

P 

159.3* 

43.1 

184.6 

54.9 

145.4 

41.8 

Disassemble  M60  Machinegun 

M 

140.8 

32.8 

169.8 

45  0 

149.9 

40.9 

P 

205.6* 

63.8 

200.8 

49.7 

177.9 

42.2 

Assemble  M60  Machinegun 

M 

189.5 

49.4 

195.6 

38.2 

169.5 

43.3 

P 

97.9 

42.6 

101.8 

58.0 

92.6 

49.4 

Disassemble  Breech 

M 

96.4 

63.1 

107.8 

77.6 

82.7 

36.3 

P 

166.1 

126.4 

129.1* 

91.3 

127.7 

72.0 

Assemble  Breech 

M 

138.4 

91.0 

171.7 

96.4 

132.4 

77.3 

P 

112.9 

35.3 

109.5 

44.0 

106.8 

36.7 

Install  Stakes 

M 

109.8 

34  4 

116.9 

38.8 

105.7 

36.3 

P 

141 .9 

32.3 

148.8 

39.0 

158.7* 

39.9 

Establish  Communication 

M 

139.1 

33.8 

148.0 

45.1 

143.6 

40.5 

*  Difference  between  means  significant  at  05  level. 
P  =  Proficiency:  M  =  Mastery 
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Table  19 

Recency  Estimates  for  Task  Performance 
Prior  to  Retention  Tests 


Retention  1 
(n  *  60) 

Retention  2 
(n  »  90) 

Retention  3 
(n  *  114) 

Tasks 

%  Soldiers 
Reporting 
Individual 
Training 
Since  Last 
Test 

%  Soldiers 
Reporting 
ARTEP  or 

Live  Fire 
Since  Last 
Test 

%  Soldiers 
Reporting 
Individual 
Training 
Since  Last 
Test 

%  Soldiers 
Reporting 
ARTEP  or 

Live  Fire 
Since  Last 
Test 

%  Soldiers 
Reporting 
Individual 
Training 
Since  Last 
Test 

%  Soldiers 
Reporting 
ARTEP  or 
Live  Fire 
Since  Last 
Test 

Measure  Quadrant 

60.0 

25.0 

73.5 

28.7 

48.3 

9.8 

Set/Lay  Cannon 

55.0 

21.6 

71.1 

29.5 

47.4 

8.1 

Sight  Target 

53.3 

5.0 

65.1 

29.5 

48.6 

8.1 

Micrometer  Test 

46.7 

20.0 

63.5 

24.1 

29.8 

7.1 

Boresight  Telescope 

55.0 

11.7 

66.7 

11.4 

46.7 

6.3 

Emplace/Recover 

Aiming  Posts 

58.3 

28.4 

72.3 

26.7 

50.9 

10.2 

Emplace  Collimator 

55.0 

26.6 

81.9 

33.0 

49.1 

10.1 

Recover  Collimator 

55.0 

26.6 

81.9 

33.0 

49  1 

10.1 

Select  Ammunition 

66.6 

46.7 

72.2 

28.4 

51  4 

11.7 

Select  Powder 

66.6 

46.7 

72.2 

28.4 

.  51.4 

11.7 

Set  Fuze 

66.6 

46.7 

72.2 

28.4 

51.4 

11.7 

Clear  M60  Machinegun 

53.3 

15.0 

80.9 

22.5 

49.1 

7.9 

Disassemble  M60 

Machinegun 

53.3 

15.0 

80.9 

22.5 

49.1 

7.9 

Assemble  M60  Machinegun 

53.3 

15.0 

80.9 

22.5 

49.1 

7.9 

Function  Check 

M60  Machinegun 

53.3 

15.0 

80.9 

22.5 

49.1 

7.9 

Read  TM-Breech 

53.3 

20.0 

76.5 

30.7 

51.3 

10.8 

Disassemble  Breech 

53.3 

20.0 

76.5 

30.7 

51.3 

10.8 

Assemble  Breech 

28.3 

20.0 

44.8 

30.7 

34.2 

10  8 

Function  Check  Breech 

28.3 

20.0 

44.8 

30.7 

34.2 

10.8 

Install  Stakes 

35.0 

16.7 

59.3 

23.9 

28.9 

9.8 

Establish  Communication 

35.0 

16.7 

59.3 

23.9 

28.9 

9.8 

Record  Information 

35.0 

16.7 

59.3 

23.9 

28.9 

9.8 
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In  the  interval  between  Acquisition  and  Retention  1, 
approximately  50%  of  the  soldiers  reported  receiving 
individual  training  on  all  tasks.  This  percentage  rose  to 
approximately  70%  between  Retention  1  and  Retention  2,  and 
fell  to  approximately  455  between  Retention  2  and 
Retention  3.  Soldiers  also  reported  practice  on  ARTEP  or 
live-fire  exercises  between  retention  test  points  although 
the  percentages  were  lower  than  for  individual  training. 

The  consistency  across  tasks  would  be  expected  for  soldiers 
from  COHORT  units. 

The  relatively  similar  amounts  of  intervening  practice 
across  tasks  preclude  any  straightforward  analysis  of 
effect  of  practice  on  retention  performance.  Furthermore, 
practice  may  interact  with  several  other  variables  (e.g., 
training  condition).  We  postpone  discussion  of  these  more 
complex  effects  until  Chapter  IV. 

Soldier  Abilities 

We  conducted  analyses  to  assess  whether  individual 
abilities  of  soldiers,  as  measured  by  the  ASVAB,  were 
related  to  performance.  First,  we  computed  correlations 
between  each  of  the  ASVAB  scores  and  each  of  the 
performance  measures  for  all  tasks  at  each  test  phase.  As 
was  the  case  in  Year  II,  although  there  were  many 
significant  correlations,  no  particular  patterns  emerged. 
Thus,  we  decided  to  explore  in  detail  only  one  of  the  ASVAB 
scores,  namely  the  Field  Artillery  (FA)  composite. 

The  correlations  between  FA  scores  and  accuracy  and 
time  performance  data  are  presented  in  Tables  20  through 
22.  The  correlations  found  at  this  bivariate  level  of 
analysis  do  not  suggest  that  FA  scores  are  strongly  related 
to  performance  accuracy.  However,  as  shown  in  Table  22, 
there  does  appear  to  be  a  relationship  between  FA  and  task 
performance  time,  especially  at  Retention  1.  Performance 
on  six  of  the  fourteen  timed  tasks  is  significantly  related 
to  FA,  while  the  correlations  for  four  other  tasks  approach 
significance.  There  also  are  several  significant 
correlations  between  FA  and  performance  time  at  Retention  2 
and  Retention  3.  As  we  have  posited  previously, 
performance  at  these  latter  two  tests  may  have  been 
affected  by  several  (possibly  interacting)  variables; 
before  we  would  conclude  that  FA  is  unrelated  to 
performance  over  extended  periods  of  time,  we  should 
examine  results  from  more  powerful  statistical  techniques 
capable  of  unravelling  complex  effects.  This  is  done  in 
Chapter  IV. 
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Table  2 0 

Correlation  Between  ASVAB  Score  on  Field  Artillery 
and  Percentage  of  Soldiers  GO/NO  GO  Across  All  Tests 


Acquisition 

Retention  1 

Retention  2 

Retention  3 

TASKS 

(N-136) 

(N=55) 

(N=86) 

(N=107) 

Measure  Quadrant 

-.01 

.06 

.24* ** *** 

.17 

Set/Lay  Cannon 

.10 

.12 

-.04 

-.16* 

Sight  Target 

o 

o 

• 

-.11 

.03 

-.08 

Micrometer  Test 

.01 

.06 

.04 

.11 

Boresight  Telescope 

.06 

.09 

.21* 

.11 

Emplace/Recover 

Aiming  Posts 

-.14* 

.11 

.03 

.02 

Emplace  Collimator 

.07 

-.06 

.01 

.10 

Recover  Collimator 

-.05 

.15 

.14 

.25 

Select  Ammunition 

-.00 

-.12 

.02 

.00 

Select  Powder 

-.01 

*** 

*** 

-.12 

Set  Fuze 

t 

• 

o 

*** 

.17 

** 

Clear  M60  Machinegun 

i 

• 

o 

UJ 

-.02 

.13 

.16 

Disassemble  M60 

Machinegun 

1 

• 

o 

Ui 

.03 

.03 

.11 

Assemble  M60  Machinegun 

1 

• 

o 

fo 

.32 

.21* 

.15 

Function  Check  M60 
Machinegun 

.02 

.00 

.02 

*** 

Read  TM- Breech 

-.07 

.19 

.12 

.18* 

Disassemble  Breech 

.12 

.29* 

.10 

.04 

Assemble  Breech 

.14* 

.35* 

.16 

.09 

Function  Check  Breech 

.07 

-.07 

.12 

.18* 

Install  Stakes 

.11 

-.16 

.02 

.14 

Establish  Communication 

.20* 

.14 

.07 

.18* 

Record  Information 

-.02 

• 

1 

.20* 

.05 

*  Significant  it  .05  level. 

**  Fuze  not  available  for  testing. 

***  100%  of  soldiers  passed  the  task. 
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Table  21 

Correlation  Between  AVSAB  Score  on  Field  Artillery 
and  Percentage  of  Steps  Across  All  Tests 


Acquisition  Retention  1  Retention  2  Retention  3 


TASKS 

(N=»136) 

(N=55) 

(N=86) 

(N=107) 

Measure  Quadrant 

-.01 

.08 

.24* 

.18* 

Set/Lay  Cannon 

.10 

-.04 

.04 

-.05 

Sight  Target 

.00 

-.03 

.02 

-.01 

Micrometer  Test 

.01 

.05 

.08 

-.05 

Boresight  Telescope 

.06 

.18 

.19* 

.14 

Emplace/Recover 

Aiming  Posts 

-.14* 

.13 

.03 

-.02 

Emplace  Collimator 

.07 

.06 

-.01 

.08 

Recover  Collimator 

-.05 

.14 

.14 

.  25* 

Select  Ammunition 

.00 

.06 

.17 

.12 

Select  Powder 

-.01 

*** 

*** 

-.12 

Set  Fuze 

-.04 

*** 

.20* 

** 

Clear  M60  Machinegun 

-.03 

-.10 

.13 

.16 

Disassemble  M60 

Machinegun 

-.05 

.02 

-.03 

.09 

Assemble  M60  Machinegun 

-.02 

.07 

.06 

.19 

Function  Check  M60 

Machinegun 

.02 

.00 

.07 

*** 

Read  TM-Breech 

-.07 

.19 

.12 

.18* 

Disassemble  Breech 

.12 

.15 

.07 

.03 

Assemble  Breech 

.14* 

.18 

.18* 

.12 

Function  Check  Breech 

.07 

.08 

.16 

.10* 

Install  Stakes 

.11 

-.16 

.01 

.09 

Establish  Communication 

.20* 

-.19 

.05 

.12 

Record  Information 

-.02 

-.11 

.17 

.04 

*  Significant  at  .05  level. 

**  Fuze  not  available  for  testing. 
***  1002  of  soldiers  passed  the  test. 
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Table  22 

Correlation  Between  ASVAB  Score  on  Field  Artillery 
and  Mean  Time  Across  All  Tests 


TASKS 

Measure  Quadrant 

Set /Lay  Cannon 

Sight  Target 

Micrometer  Test 

Boresight  Telescope 

Emplace/Recover 
Aiming  Posts 

Emplace  Collimator 

Recover  Collimator 

Disassemble  M60 
Machinegun 

Assemble  M60  Machinegun 
Disassemble  Breech 
Assemble  Breech 
Install  Stakes 
Establish  Communication 


*  Significant  at  .05  level. 


Acquisition  Retention  1  Retention  2  Retention  3 
<N=136)  (iJ»55)  (N-86)  (N=l07) 


.00 

-.18 

-.16 

-.23* 

-.03 

-.34* 

-.12 

.04 

-.10 

-.27* 

-.22* 

-.14 

.00 

-.09 

-.24* 

-.17* 

-.04 

-.22 

-.29* 

™ .  07 

-.10 

-.20 

-.06 

-.07 

-.11 

.03 

-.04 

-.10 

-.17* 

.07 

.12 

-.07 

-.06 

-.23* 

-.13 

-.11 

i 

« 

O 

-.29* 

-.20 

-.10 

-.02 

-.30* 

-.19 

-.08 

-.22* 

-.33* 

-.18 

-.24* 

.06 

.17 

.04 

-.25* 

.03 

-.13 

.13 

.05 

Duty  Position  Assignment 

In  a  combat  environment.  Artillery  crewmen  are 
assigned  to  very  specific  jobs.  Each  soldier  has  a 
well-defined  position  where  specific  tasks  are  performed. 

On  the  other  hand,  in  a  (hypothetical)  training 
environment,  individual  crewmen  are  responsible  for  all 
crew  position  tasks.  Some  units  systematically  crosstrain 
crewmen  —  all  soldiers  are  given  training  and  practice  on 
all  crew  positions  —  while  other  units  stress  individual 
skills  at  the  crewman's  primary  duty  position. 

Since  many  of  the  tasks  we  selected  were  specific  to 
individual  crew  positions,  we  examined  the  effects  of  two 
additional  variables  on  performance:  the  soldiers'  primary 
duty  position  and  whether  or  not  the  soldier  had  been 
crosstrained  on  other  positions.  We  distinguished  between 
Gunner,  Assistant  Gunner,  and  Cannoneer  #1  positions  and 
the  other  Cannoneer  positions.  During  the  seven  month 
testing  period,  most  (76.1%)  soldiers  did  not  change  their 
primary  duty  positions.  B'orty-nine  soldiers  (37.6%)  were 
assigned  one  of  these  three  positions  as  their  primary  duty 
position.  An  additional  48  (36.9%)  soldiers  were 
crosstrained  in  these  positions.  The  influence  of  these 
assignments  will  be  examined  in  Chapter  IV. 

Error  Performance 

It  is  useful  to  examine  the  specific  errors  made  by 
soldiers.  The  step-by-step  performance  is  useful  to  review 
since  it  allows ■ determination  of  the  steps  that  are  most 
difficult  and  therefore  should  be  emphasized  during 
training . 

Below  v/e  will  summarize  the  most  frequent  errors  made 
on  each  task. 

Measure  Quadrant  (2  steps):  Results  from  the 
Acquisition  and  Retention  tests  revealed  soldiers  had  to 
remember  to  center  the  bubbles  without  moving  the  cannon 
tube.  This  first  error  reveals  a  conceptual 
misunderstanding:  if  you  move  the  tube,  you  change  the 

quadrant.  Most  soldiers  remembered  that  they  had  to  center 
the  bubbles,  but  many  did  this  by  simply  mcving  the  tube. 
Second  was  the  physical  problem  of  actually  centering  the 
bubbles  accurately  within.  15  seconds,  a  relatively 
demanding  skill. 
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Set/Lay  Cannon  (5  steps):  Errors  occurred  most 
commonly  on  two  steps  —  repeating  the  quadrant  command  and 
centering  the  longitudinal  bubble  by  elevating  or 
depressing  the  cannon  tube.  The  former  is  a  procedural 
step  that  soldiers  may  have  considered  unnecessary  in  our 
test  situation;  the  latter  is  again  a  relatively  demanding 
physical  skill,  given  the  time  constraints  of  the  task. 

Sight  Target  (3  steps):  Results  indicated  two  sources 
of  difficulty.  Two  of  the  three  steps  are  procedural 
(repeating  the  range  command  and  repeatedly  announcing 
"Set"  after  sighting  on  target);  soldiers  again  may  have 
considered  these  procedures  as  unnecessary.  The  other 
problem  is  that  soldiers  had  to  mentally  determine  the 
correct  mil  line  to  put  on  the  target.  This  involved 
referring  to  a  chart  attached  to  the  howitzer  that  gives 
appropriate  mils  for  different  ranges.  Soldiers  had  to 
remember  to  refer  to  the  chart;  furthermore,  on  our  test  we 
forced  them  to  extrapolate  between  two  ranges  listed  on  the 
chart  (e.g.,  the  chart  lists  ranges  of  1000  and  1200 
meters;  soldiers  were  given  a  range  of  1100).  Some 
soldiers  forgot  to  refer  to  the  chart;  others  did,  but  used 
the  wrong  mil  line. 

Micrometer  Test  (8  steps):  The  pattern  of  errors 
during  the  retention  tests  have  led  us  to  speculate  that 
soldiers  may  have  "overprocedural ized"  this  relatively 
straightforward  task.  The  gunner's  quadrant  has  an  index 
arm  calibrated  in  mils  and  a  micrometer  knob  calibrated  in 
tenths  of  mils.  The  micrometer  test  is  to  set  either  the 
arm  or  the  knob  on  zero  and  tne  other  to  one  mil,  level  the 
quadrant,  reverse  the  settings,  and  see  if  the  quadrant  is 
still  level.  Soldiers  seem  to  learn  this  task  in  a 
strictly  "rote"  fashion.  For  example,  many  soldiers  set 
both  the  knob  and  the  arm  to  zero,  leveled  the  quadrant, 
and  reset  both  the  knob  and  the  arm  to  one  mil  (and 
reported  that  the  quadrant  was  not  level  and  therefore 
malfunctioning).  While  they  memorized  the  procedure,  they 
did  not  have  sufficient  conceptual  understanding  to  avoid 
these  errors. 

Boresight  Telescope  (8  steps):  This  task  was 
unfamiliar  to  most  of  the  tested  soldiers  (fewer  than  15% 
knew  how  to  do  the  task  at  all  when  initially  tested). 
Furthermore,  the  pattern  of  errors  suggested  that  soldiers 
viewed  it  as  a  completely  arbitrary  procedure  witn  no 
feedback  that  could  help  them  determine  if  they  were  doing 
the  task  correctly.  Errors  of  omission  and  steps  out  of 
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sequence  were  common;  in  particular,  the  totally  arbitrary 
step  of  subtracting  2.3  mils  from  the  elevation 
(necessitated  by  the  offset  bore  hole  of  the  M198)  was 
consistently  forgotten  or  miscalculated. 

Emplace/recover  Aiming  Posts  (8  steps);  During  the 
retention  tests  the  most  common  error  was  that  soldiers 
failed  to  position  the  near  post  approximately  50  meters 
from  the  howitzer.  Again,  this  may  have  seemed  to  be  an 
arbitrary  distance  for  soldiers. 

Emplace  Collimator  (16  steps):  Performance  was 
excellent  on  this  task;  soldiers  passed  over  90%  of  all 
steps.  Only  in  Retention  2  were  any  steps  passed  by  fewer 
than  90%  of  the  soldiers.  The  problem  steps  were  the 
procedural  ones  of  opening  the  lens  cover  before  starting 
adjustment,  and  failure  to  center  the  azimuth  adjustment 
knob. 

Recover  Collimator  (7  steps):  The  procedural  step  of 
tightening  the  elevation  clamping  knob  was  the  only  step 
which  any  soldiers  missed  during  all  three  Retention  tests. 

Select  Ammunition  (10  steps):  The  task  included 
selecting  both  the  correct  projectile  and  the  correct  fuze 
required  by  each  fire  command.  In  one  sense,  this  is 
analogous  to  a  paired-associate  verbal  learning  task,  where 
there  are  specific  (arbitrary)  responses  to  each  command. 
Soldiers  must  memorize  the  fact  that,  for  example,  the  M514 
fuze  is  the  correct  response  to  "Fuze  Victor  Tango".  In 
addition,  the  fuzes  themselves  do  not  give  any  clue  to 
their  use. 

Select  Powder  (2  steps):  Neither  step  provided  any 
consistent  difficulty  for  the  soldiers. 

Set  Fuze  (2  steps):  Again,  no  real  difficulty  was 
found  with  either  step. 

Clear  M60  Machinegun  (4  steps):  All  steps  were  passed 
by  at  least  86%  of  the  soldiers  at  each  Retention  test. 

The  step  which  was  passed  by  the  fewest  soldiers  varied  at 
each  test. 

Disassemble  M60  Machinegun  (6  steps):  The  most  common 
errors  were  that  soldiers  did  not  separate  the  operating 
rod  from  the  bolt  or  did  not  separate  the  spring  and  the 
spring  guide.  Again,  these  steps  may  have  been  viewed  as 
unnecessary  by  the  soldiers. 
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Assemble  M60  Machinegun  (7  steps):  "Replace  cover, 
tray,  and  hanger  group"  was  the  most  common  error. 

Soldiers  must  remember  that  the  hinge  pin  goes  on  the  right 
side  and  the  latch  goes  on  the  left;  there  is  nothing  on 
the  machinegun  itself  that  could  provide  a  cue  to  the 
soldier  for  these  steps. 

Function  Check  M60  Machinegun  (5  steps):  No  steps 
presented  a  consistent  problem. 

Read  TM-Breech  (1  step):  The  entire  task  was  to  open 
the  training  manual  to  the  section  on  breech  maintenance. 

At  Acquisition  only  21.4%  of  the  soldiers  passed  this  step, 
perhaps  indicating  that  this  was  not  a  common  practice. 

This  view  was  reinforced  when  we  noticed  that  it  was  nearly 
impossible  to  put  the  TM  in  a  position  where  it  would 
remain  open  and  where  the  soldier  could  read  it  while  using 
both  hands  to  disassemble  or  assemble  the  bulky  breech 
mechanism. 

Disassemble  Breech  (5  steps):  No  steps  presented  any 
particular  problem. 

Assemble  Breech  (8  steps):  The  step  where  most  errors 
were  made  was  to  assemble  the  obturator  so  that  the  split 
rings  were  180  degrees  apart.  Again,  there  is  no  apparent 
reason  why  the  rings  should  be  assembled  this  way; 
soldiers  apparently  forgot  this  arbitrary  procedure. 

Function  Check  Breech  (2  steps):  Soldiers  had  only 
slight  problems  with  each  of  these  steps. 

Install  Stakes  (4  steps):  The  most  difficult  part  of 
this  task  was  making  the  final  alignment  between  the  gun 
guide  stake  and  pantel  marking.  This  involves  some  degree 
of  physical  skill  and  a  steady  hand. 

Establish  Communication  (9  steps):  Two  steps  — 
"Install  the  field  telephone  in  an  area  near  the  pantel 
stake  and  clear  of  whore  the  howitzer  would  be  positioned," 
and  “Turn  IMT-EXT  switch  to  INT"  —  were  passed  by  the 
fewest  numoer  of  soldiers.  Both  of  these  steps  are 
non-cued  parts  of  fairly  lengthy  procedures. 

Record  Information  (2  steps):  The  step  of  pacing  off 
the  distance  from  the  aiming  circle  to  the  Pantel  marking 
stake  and  recording  it  was  frequently  missed.  It  may  have 
been  considered  unnecessary  to  the  soldiers. 


Summary  of  Retention  Performance 

The  preceding  analyses  showed  that  retention  losses 
did  occur  between  Acquisition  and  Retention.  The  tasks 
showing  the  greatest  losses  were  largely  those 
characterized  as  most  difficult  at  Acquisition.  However, 
we  found  that  soldiers  performed  better  on  over  half  (14 
out  of  22)  of  the  tasks  at  Retention  3  relative  to  the 
previous  two  testings.  In  fact,  on  over  half  of  the  tasks 
performance  levels  at  Retention  3  were  greater  than  those 
demonstrated  at  Retention  1.  Percentages  of  soldiers  "GO" 
at  Retention  2  were  the  lowest  for  any  of  the  three  test 
points.  The  average  number  of  tasks  passed  by  soldiers  at 
Retention  2  was  12.  This  compares  with  an  average  of  16 
for  both  Retention  1  and  3  and  an  average  of  13  at  the 
first  trial  of  Acquisition. 

Retention  1  presented  the  only  clear  differences 
between  training  conditions  and  performance  measures. 
Generally  we  found  that  a  greater  percentage  of  Mastery 
soldiers  passed  the  tasks.  This  difference  washed  out  at 
the  subsequent  retention  points. 

We  reviewed  Percentage  of  soldiers  "GO"  on  each  task 
step.  For  the  most  part  only  one  or  two  steps  per  task 
presented  problems  for  the  soldiers.  However,  on  some  of 
the  more  difficult  tasks,  such  as  Sight  Target  and 
Disassemble  Breech,  each  of  the  steps  presented  the  same 
level  of  difficulty. 

Performance  times  provided  some  interesting 
distinctions.  Retention  losses  occurred  between  the  last 
trial  of  Acquisition  and  the  first  Retention  testing.  In 
other  words,  more  time  was  required  for  task  completion  at 
Retention  testings  than  at  the  end  of  Acquisition.  These 
losses  tended  to  be  of  the  same  magnitude,  regardless  of 
the  amount  of  time  between  the  two  tests.  The  anomaly  to 
this  finding  was  that  "Install  Stakes"  and  "Establish 
Communication"  took  less  time  to  complete  at  the  Retention 
test  than  at  the  last  trial  of  Acquisition. 

Other  factors,  e.g.,  primary  duty  position,  ASVAB 
scores,  and  recency  of  task  performance,  were  also 
reviewed.  Nearly  three-fourths  of  the  sample  were  either 
assigned  to  or  crosstrained  in  the  Gunner,  Assistant 
Gunner,  or  Cannoneer  #1  positions.  With  minor  exceptions, 
FA  scores  and  recency  of  task  performance  prior  to  each 
testing  did  not  provide  any  strong  explanations  for  task 
performance . 


In  Chapter  IV,  we  will  consider  the  combined  effects 
of  these  factors  on  performance,  controlling  for  individual 
differences.  Before  this,  however,  we  will  present  a 
discussion  of  the  User's  Decision  Aid  (UDA),  since  the  task 
difficulty  ratings  generated  by  the  UDA  were  used  as 
predictors  of  performance. 


III.  THE  USER’S  DECISION  AID  (UDh) 


The  User's  Decision  Aid  developed  in  Year  II  provided  a 
useful  tool  for  predicting  task  performance  based  on  users' 
ratings.  During  Year  III,  we  had  three  major  goals  for  the 
further  development  of  the  UDA:  to  build  a  firm  statistical 
and  empirical  foundation  to  support  the  algorithm  using  data 
collected  in  Year  II,  to  improve  the  UDA's  usefulness  and 
range  of  applicability,  and  to  validate  UDA  predictions 
against  Year  III  13B  data. 

Toward  the  first  goal  we  developed  and  implemented  a 
procedure  to  empirically  determine  the  values  for  each  answer 
option  of  the  UDA  questions  that  best  predict  actual 
retention  performance.  We  also  developed  an  empirically 
based  rule  that  generates  proficiency  predictions  for  any 
time  interval. 

Toward  the  second  goal ,  we  improved  the 
comprehensibility  of  the  rating  procedure,  increased  the 
UDA's  applicability  to  other  sets  of  tasks,  and  incorporated 
other  factors,  such  as  time  constraints,  into  the  algorithm, 
We  will  discuss  these  developmental  activities  in  this 
chapter.  Toward  the  third  goal,  we  conducted  a  series  of 
multiple  regression  analyses.  This  effort  in  described  in 
Chapter  IV. 


UDA  Development 

In  Year  II,  we  made  three  arbitrary  assumptions  in  the 
construction  of  the  UDA.  First,  the  values  assigned  to  each 
answer  option  (e.g.,  on  Question  2,  a  "good"  job/memory  aid 
equals  two  points,  a  "poor”  job/memory  aid  equaJs  three 
points)  were  not  derived  empirically.  Second,  the  relative 
weights  for  the  questions  and  the  rule  for  combining 
questions  (i.e.,  equal  weights  and  a  straight  addition  rule) 
were  also  arbitrary.  Third,  the  rule  that  was  used  to 
estimate  the  rate  of  proficiency  loss  —  the  proportional 
decrease  in  the  percentage  of  soldiers  able  to  do  a  task  over 
time  —  while  not  completely  arbitrary,  was  not  statistically 
derived.  (The  value  of  2.5  percent  loss  per  month  per  UDA 
scale  score  point  was  derived  from  the  two-month  retention 
data  only;  extrapolations  to  four-  and  six-month  retention 
intervals  were  made  but  not  evaluated.) 

Although  our  UDA  assumptions  "worked"  for  the  Year  II 
results  (i.e.,  the  task  ratings  correlated  highly  with 
retention  performance),  we  did  not  have  an  explicit  and 
defensible  rationale  for  assigning  values  to  answer  options. 
The  major  reason  for  this  w~s  that  we  did  not  have  a 
sufficient  "pool"  of  tasks  to  allow  us  to  conduct  the 
regression  analyses  necessary  to  statistically  determine 
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"best-fif'  values.  Simply  stated,  the  problem  was  that  we 
had  too  few  cases  (i.e.,  tasks)  for  too  many  predictors. 

To  increase  the  number  of  cases,  we  combined  the  11B 
task  retention  data  with  retention  data  collected  from  a 
sample  of  Basic  Combat  Training  ( BCT )  tasks.  Briefly,  during 
Year  III,  project  staff  were  involved  in  a  data  collection 
effort  at  Fort  Jackson,  SC.  Retention  scores  for  135 
soldiers  on  BCT  tasks  were  collected  about  two  months  after 
the  soldiers  had  completed  BCT.  These  data  increased  our 
pool  to  a  total  of  54  tasks:  27  from  MOS  11B  and  27  from  BCT. 

To  decrease  the  number  of  predictors,  we  conducted  a 
factor  analysis  of  the  task  dimensions  comprising  the  UDA. 
Based  on  the  results  of  that  analysis,  we  were  able  to  reduce 
the  number  of  UDA  task  characteristic  predictors  to  five. 

Although  the  resulting  data  set  based  on  criterion  data 
points  for  54  tasks  and  five  predictors  was  still  not  ideal, 
it  enabled  us  to  conduct  a  series  of  regression  analyses.  In 
support,  of  these  analyses,  the  first  activity  was  to  obtain 
UDA  ratings  for  each  of  the  54  tasks  in  the  criterion  pool. 

We  collected  these  ratings  from  eight  raters,  using  the 
Year  II  scales  and  questions.  Our  criterion  (performance) 
measures  were  Percent  Soldiers  GO  (11B,  BCT  tasks)  and 
Percent  Steps  GO  (11B  tasks)  at  two-month  retention  testing. 
The  analyses  conducted  on  these  data  resulted  in  empirically 
based  scale  values  for  the  answer  options  within  each  UDA 
question  (i.e.,  dimension).  They  also  provided  regression 
v/eights  for  each  UDA  question. 

Table  23  illustrates  the  steps  undertaken  to  determine 
question  weights,  scale  values  for  each  answer  option,  and 
the  rule  for  calculating  an  overall  task  score.  Details  of 
the  procedures  for  each  step  are  described  in  the  following 
sections . 

Step  Is  Find  modal  response  from  raters.  To  simplify 
the  analysis,  we  needed  to  select  a  single  set  of  responses 
from  the  eight  raters.  High  interrater  reliability  had 
already  been  established  in  the  Year  II  UDA,  so  the  pooling 
of  responses  across  raters  could  be  made  without  sacrificing 
predictive  power. 

Our  selection  procedure  consisted  of  finding  the  modal 
response  to  each  question  for  each  task  from  the  eight 
raters.  This  could  be  thought  of  as  a  "majority  rules" 
approach.  One  judge  was  chosen  to  be  the  tie-breaker  in 
cases  where  more  than  one  mode  existed.  This  approach  was 
chosen  for  three  reasons:  first,  mean  ratings  would  not  be 
useful,  since  we  were  working  with  discrete  scale  values 
within  each  question.  Second,  the  mode  excludes  values  at 
the  extremes  of  the  distribution;  and  third,  the  different 
judges  represented  different  perspectives  on  the  ratings  • — 
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These  questions  have  the  same  weight,  since  they  were  combined  in  the  regression  analysis. 


some  judges  were  "task"  experts,  while  others  were  "UDA" 
experts,,  We  displayed  the  modal  ratings  obtained  by  this 
procedure  in  a  matrix  defined  by  the  UDA  questions  (columns) 
and  by  the  54  tasks  (rows).  This  matrix  is  referred  to  as 
the  "mode  matrix."  The  modal  ratings  for  the  UDA  scales 
(reduced  to  eight  by  combining  the  mutually  exclusive 
questions  one  and  two)  are  shown  in  Table  23  for  the  Visual 
Signals  task. 

Step  2:  Replace  modes  with  corresponding  cell  means. 

This  .step  was  performed  by  inputting  the  mode  matrix  to  an 
analysis  of  variance  (ANOVA)  program.  We  conducted  a 
separate  ANOVA  for  each  of  the  eight  questions.  The 
dependent  variable  for  each  ANOVA  was  the  Percentage  of 
Soldiers  GO  measure.  The  independent  variable  was  the  answer 
options  for  each  question  used  as  a  grouping  variable.  For 
example,  all  tasks  having  a  modal  rating  of  "1"  on  a  given 
question  were  assigned  to  the  “1"  Group;  those  having  a  modal 
rating  of  "2"  were  assigned  to  the  "2”  Group,  etc.  We 
obtained  eight  separate  F  statistics. 

Cell  means  (i.e.,  the  means  of  the  criterion  retention 
measure)  were  generated  for  each  answer  option  for  each 
question  (a  total  of  29  possible  answer  options).  For 
example,  the  first  cell  mean  would  be  the  average  of  the 
t.wo-month  retention  performance  measure  for  those  tasks  whose 
modal  UDA  rating  included  the  first  category  on  the  first 
question,  and  so  on  for  all  the  response  options  in  all  the 
questions. 

The  F  statistic  describes  the  significance  of  the 
relationship  between  the  UDA  modal  scores  and  the  retention 
performance  measure.  For  those  questions  with  significant  F 
values,  the  pattern  of  cell  means  was  as  expected;  tasks  with 
higher  modal  ratings  (e.g.,  those  that  were  more  "difficult") 
had  lower  mean  performance  in  terms  of  the  two-month 
retention  measure. 

Three  of  the  questions  had  nonsignificant  F  values.  On 
Question  1,  the  availability  and  quality  of  job  aids,  and  on 
Question  8,  the  physical  demands  of  the  task,  the  modal 
ratings  were  highly  skewed.  Job  aids  were  not  used  in  49  of 
the  54  tasks;  only  one  task  placed  considerable  physical 
demands  on  soldiers.  In  these  cases  the  Group  sizes  were  too 
small  to  support  meaningful  ANOVAs.  Question  3,  the 
sequential  requirements  of  the  task,  is  in  fact  a  "new" 
question;  it  was  not  contained  in  the  Year  II  UDA.  As  will 
be  discussed  further  below,  this  question  was  added  due  to 
feedback  from  UDA  users. 

It  happened  that  there  were  no  tasks  in  our  sample  that 
had  "very  complex"  mental  requirements  or  that  had 
"excellent"  job  aids.  For  these  cases,  and  for  options  for 
the  questions  mentioned  above,  weights  were  assigned 


arbitrarily.  Thus,  for  "excellent"  job  aids,  we  decided  that 
the  assigned  weight  should  be  equivalent  to  the  sum  of  the 
weights  of  the  questions  that  would  be  skipped  if  an 
excellent  job  aid  was  in  fact  used.  This  weight,  and  others 
arbitrarily  assigned,  could  not  be  evaluated  with  the  present 
data  base;  their  evaluations  await  further  empirical 
research . 

With  the  exceptions  noted  above,  we  transformed  the  mode 
matrix,  replacing  each  entry  with  the  corresponding  cell  mean 
from  the  ANOVA.  For  example,  on  the  first  question,  a  modal 
rating  of  "2"  ("Good"  job  aids)  received  a  transformed  score 
of  "61.3,"  corresponding  to  the  mean  percentage  of  soldiers 
GO  for  all  tasks  with  "Good”  job  aids.  Similarly,  modal 
ratings  of  "3"  and  "4"  received  transformed  scores  of  "63.9" 
and  "70.5,"  respectively. 

From  the  F  statistics  associated  with  the  eight  one-way 
ANOVAs ,  it  appeared  that  most  of  the  predictive  power  of  the 
UDA  was  coming  from  four  questions.  To  explore  the 
relationships  among  questions  further,  we  conducted  a  factor 
analysis  of  the  cell-mean  matrix,  again  with  mean  two-month 
Percent  Soldiers  GO  as  the  dependent  variable. 

The  results  of  the  Factor  Analysis  supported  those  of 
the  ANOVA.  The  factor  accounting  for  the  largest  proportion 
of  variance  could  be  interpreted  as  measuring  the  cognitive 
demands  of  the  task.  Four  questions  loaded  significantly  on 
that  factor.  We  chose  to  combine  these  four  into  a  single 
independent  variable,  by  adding  their  values  together.  The 
other  four  questions  seemed  to  measure  different  dimensions 
of  the  tasks. 

Step  3:  Multiply  by  question  weights.  We  next  conducted 
a  regression  analysis  in  which  we  examined  the  relationship 
between  the  five  UDA  (predictor)  variables  and  mean  two-month 
retention  performance  across  the  54  tasks.  The  result  of 
this  regression  analysis  was  that  79%  of  the  variance  in  the 
criterion  measure  could  be  accounted  for  by  the  five 
predictor  variables  (i.e.,  the  multiple  R  *  .890).  This  can 
be  compared  to  the  proportion  of  variance  accounted  for  by 
the  Year-II  UDA,  which  was  66%  (multiple  R  =  .810). 

The  regression  analysis  also  supplied  weights  for  the 
questions.  We  used  the  B's  (unstandardized  regression 
coefficients)  as  weights  for  the  eight  questions.  For  he 
answer  options,  the  cell  means  would  be  multiplied  by  their 
corresponding  question  weights  (cf.  Table  23). 

Steps  4  and  5:  Transforming  scale  values  and  row-wise 
addition.  While  the  scale  values  produced  by  the  preceding 
steps  are  "best  fits"  to  the  data,  they  are  relatively 
difficult  to  work  with.  From  a  practical  perspective,  we 
would  rather  not  have  UDA  users  adding  positive  and  negative 
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decimals;  therefore,  we  made  a  final  set  of  transformations 
on  the  scale  values  to  improve  their  useability. 

Our  procedure  was  straightforward:  we  rounded  off  the 
weights  to  the  nearest  whole  number,  then  added  or  subtracted 
a  constant  for  each  question  so  that  the  lowest  value 
equalled  zero.  Then  the  sum  of  these  additions  and 
subtractions  was  added  to  the  regression  equation  constant. 
These  transformations  have  no  effect  on  the  regressions:  the 
regression  equation  still  accounts  for  79%  of  the  variance  in 
the  criterion  measures. 

The  final  step  is  to  add  up  the  eight  transformed  scores 
for  each  task  (across  each  row  in  the  matrix)  to  arrive  at  a 
total  UDA  score  for  the  task. 


Development  of  a  generation  rule.  The  purpose  of  the 
ur«A  is  to  generate  numerical  predictions  of  proficiency.  For 
example,  what  does  a  UDA  value  of  ”86"  mean,  and  how  can  this 
value  be  used  to  estimate  performance? 

For  one  application  —  predicting  two-month  retention 
performance  —  the  translation  of  the  UDA  value  is 
straightforward.  Since  the  option  weights  and  combination 
rule  were  developed  from  these  data,  the  two-month  retention 
estimate  is  the  UDA  value:  i.e.,  the  addition  of  the  scale 
scores,  corrected  by  a  constant;  for  the  example  in  Table  23, 
this  constant  is  equal  to  -40.  Thus,  a  UDA  value  of  "86" 
translates  into  a  prediction  of  86  -  40  *  26  percent 
proficiency  after  two  months.  (The  constant  used  in  the 
final  UDA  algorithm  is  equal  to  -80.) 

Other  applications,  such  as  predicting  four-month 
retention  performance,  involve  a  different  set  of 
considerations.  For  example,  a  demonstration  that  the  UDA 
values  derived  from  two-month  data  correlate  with  four-month 
retention  does  little  more  than  show  that  the  UDA  orders 
tasks  correctly;  it  does  not  generate  proficiency  estimates 
in  terms  of  absolute  values.  Nor  did  we  want  to  calculate  a 
different  regression  equation  for  each  time  interval  to  be 
predicted.  Thus,  we  wanted  to  incorporate  into  the  UDA  a 
function  that  would  relate  the  UDA  value  and  time  interval  to 
proficiency  scores. 

For  the  Year  II  UDA,  this  relationship  was  expressed  as 
a  negative  exponential  function,  with  the  proficiency 
prediction  decreasing  at  the  rate  of  2.5%  per  month  per  UDA 
scale  score.  The  value  of  the  constant  was  selected  to 
maximize  the  "fit"  with  the  two-month  retention  data  (MOS 
11B)  and  to  enable  us  to  generate  predictions  for  other  time 
intervals.  We  did  not  evaluate  the  accuracy  of  this  function 
for  Year-II  data  beyond  the  two-month  retention  scores, 
primarily  because  the  four-month  and  six-month  retention  data 
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did  not  show  systematic  decreases.  We  felt  that  the  Year  III 
13B  retention  data  would  be  precise  enough  to  be  used  in 
developing  and  evaluating  an  empirically-based  generation 
rule , 


We  followed  an  iterative  procedure:  we  hypothesized  a 
function,  generated  predictions  for  two-  and  four-month 
retention,  compared  actual  and  predicted  results, 
hypothesized  a  different  function,  etc.  Comparisons  were 
made  in  terms  of  the  obtained  correlation  and  absolute  and 
arithmetic  error  between  actual  and  predicted  scores.  The 
only  constraints  we  placed  on  the  hypothesized  function  was 
that  it  be  asymptotically  zero  along  the  "performance"  axis. 

We  tested  several  dozen  hypotheses.  The  best  fitting 
function  —  the  one  that  had  the  highest  correlation  and 
smallest  absolute  and  algebraic  difference  —  is  as  follows: 


Predicted  proficiency 
for  week  Y 


100  X 


Y 


The  rule  is:  divide  the  UDA  value  (after  subtracting 
the  constant)  by  100;  take  the  eighth  root;  raise  the  result 
to  the  Yth  power;  multiply  by  100.  For  two  months,  Y  =  8, 
and  the  formula  reduces  to  just  the  UDA  score,  as  mentioned 
above . 


When  we  used  this  rule  to  generate  predictions  for  two- 
and  four-month  proficiency  and  compared  the  predictions  to 
the  actual  data  (i.e.,  138  Retention  1  and  Retention  2  GO/NO 
GO  percentages  for  22  tasks),  we  obtained  a  correlation  of 
r  =  0.83;  the  mean  predicted  score  was  67.26  (over  all 
tasks),  while  the  actual  mean  was  66.75.  Thus,  the  absolute 
error  was  less  than  one  half  of  one  percent  per  task. 


Revisions  of  the  UDA 

One  of  our  goals  during  Year  III  was  to  improve  the 
comprehensibility  and  useability  of  the  UDA  rating  procedure. 
During  the  year,  we  had  several  opportunities  to  interact 
with  actual  and  potential  users  of  the  UDA.  Primarily  as  a 
result  of  these  interactions,  several  changes  were  made  in 
the  questions  and  answer  options. 


For  example,  we  found  that  users  had  difficulty  with 
Question  Four  ("Do  the  steps  tend  to  follow  a  natural 
sequence  in  which  completing  one  step  suggests  what  the  next 
step  should  be?").  Therefore,  we  decided  to  separate  this 
into  two  "simpler"  questions:  "Are  the  steps  in  the  task 
required  to  be  performed  in  a  definite  sequence?'5  and  "Does 
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the  task  have  any  built-in  logic  so  that  you  know  you  are 
doing  it  correctly?"  Similarly,  we  modified  the  language  of 
several  of  the  other  questions  and  answers.  While  we 
conducted  no  formal  reliability  analyses,  users  of  the 
different  versions  of  the  UDA  found  the  rewritten  version  to 
be  the  more  comprehensible. 

Another  change  resulted  from  statistical  analyses  of  the 
MOS  13B  retention  data.  We  found  that  the  predicted  and 
actual  scores  differed  significantly  for  a  number  of  tasks, 
specifically  those  tasks  with  time  limits.  We  were 
predicting  higher  levels  of  proficiency  than  were  actually 
observed  for  tasks  in  which  soldiers  were  exceeding  the  time 
standards.  For  example,  many  soldiers  took  longer  than  the 
specified  time  limit  when  measuring  the  quadrant.  Thus,  we 
tested  the  effect  of  adding  a  question  to  the  UDA:  "Does  the 
task  have  a  time  limit  for  its  accomplishment?" 

To  determine  weights  for  answer  options,  we  rerated  all 
11B,  13B  and  Common  Soldier  tasks,  and  repeated  Steps  1-5 
(discussed  above).  These  weights  are  used  in  the  current 
version  of  the  UDA. 


The  Current  UDA 

Below  are  the  questions,  options  and  scale  values  of  the 
current  UDA.  For  a  detailed  description  of  this  information, 
refer  to  Rose,  Radtke,  Shettel,  and  Hagman  (1984). 


Question  1.  Are  job  or  memory  aids  intended  to  be  used  in 
performing  this  task? 

Scale  Value 

1  e  Yes. 

0  •  No. 
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Question  2. 
memory  aid? 

How  would  you  rate  the  quality  of  the  iob  or 

Scale  Value 

56 

•  Excellent.  Using  the  job/memory  aid, 
a  soldier  can  do  the  task  correctly 
with  no  additional  information  or  help. 

25 

•  Very  Good.  With  the  job/memory  aid, 
a  soldier  would  need  only  a  little 
additional  information  to  complete  the 
task . 

2 

•  Marginally  Good.  Even  with  the  job/ 
memory  aid,  a  soldier  would  need 
some  additional  information  to 
complete  the  task. 

1 

o  Poor.  Even  with  the  job/memory  aid, 
a  soldier  would  need  a  great  deal  of 
additional  information  in  order  to 
complete  the  task. 

Question  3. 

How  many  steps  are  required  to  do  the  task? 

Scale  Value 

25 

•  One  step. 

14 

•  Two  to  five  steps. 

12 

•  Six  to  ten  steps. 

0 

•  More  than  ten  steps. 

Question  4. 

Are  the  steps  in  the  task  required  to  be 

performed  in 

a  definite  sequence? 

Scale  Value 

10 

•  None  are. 

5  •  All  are. 

0  •  Some  are  and  some  are  not. 
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Question  5.  Does  the  task  have  a  built-in  logic  so  that  you 
can  tell  if  you  are  doing  it  correctly? 

Scale  Value 


22 

• 

Has 

built-in 

logic 

for 

all 

steps . 

19 

• 

Has 

built-in 

logic 

for 

most 

steps . 

11 

• 

Has 

built-in 

logic 

for 

only 

a  few  steps. 

0 

• 

Has 

no  built- 

•in  logic. 

Question  6.  Does  the  task  have  a  time  limit  for  its 
completion? 

Scale  Value 

40  •  There  is  nc  time  limit. 


35 


•  There  is  a  time  limit,  but  it  is  fairly  easy 
to  meet  under  test  conditions. 


0 


«  There  is  a  time  limit  and  it  is  difficult  to 
meet  under  test  conditions. 


Question  7.  What  are  the  mental  or  thinking  requirements  of 
this  task? 

Scale  Value 


37 

28 

3 

0 


•  Almost  no  mental  requirements. 

•  Simple  mental  requirements. 

•  Complex  mental  requirements. 

•  Very  complex  mental  requirements. 
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Question  8.  How  many  facts,  terms,  names,  rules  or  ideas 
must  a  soldier  memorize  in  order  to  do  the  task? 


Scale  Value 


20 

• 

None  (or  the  job/memory  aid 
provides  all  necessary  information). 

18 

• 

A  few  (1  -  3) . 

13 

• 

Some  (4  -  8 ) . 

0 

• 

Very  many  (more  than  8). 

Question  9. 

remember? 

How 

hard  are  the  facts,  terms,  etc.,  to 

Scale  Value 

34 

• 

Not  applicable  -  there  are  none  to 
remember  or  the  job  or  memory  aid 
provides  all  of  the  needed  information. 

31 

• 

Not  hard  at  all  -  the  information  is 
simple. 

12 

• 

Somewhat  hard  -  some  of  the  information 
is  complex. 

0 

• 

Very  hard  -  the  facts,  rules,  terms, 
etc.,  are  technical  or  specific  to  the 
task  and  must  be  remembered  in  exact 
detail. 

Question  10. 

What  are  the  motor  skill  demands  of  the  task 

Scale  Value 

2 

• 

None . 

0 

• 

Small  but  noticeable  amount  of 
motor  skill  required. 

16 

• 

Considerable  amount  of  motor 
skill  needed. 

3 

• 

Very  great  demand  for  motor 
skill. 

IV 


PREDICTING  PERFORMANCE 


Introduction 

The  preceding  chapter  described  development  and 
refinement  of  a  UDA.  This  chapter  discusses  its 
validation:  Do  performance  predictions  generated  by  the  UDA 
correspond  to  empirical  results?  The  empirical  results 
used  for  this  validation  are  the  1 3B  data  presented  in 
Chapter  II,  since  they  were  not  used  in  the  analytic 
development  of  the  weights  for  the  UDA  answer  options. 

Thorough  understanding  of  the  13B  retention  data  also 
is  important.  As  already  discussed  in  Chapter  II,  there 
were  many  potential  influences  on  performance.  Some  of 
these  stemmed  from  experimental  manipulations  (e.g., 

Mastery  versus  Proficiency  training,  number  of  retention 
tests  a  soldier  took).  Others  stemmed  from  a  variety  of 
individual  difference  variables,  including  a  soldier's 
background  (e.g.,  his  ASVAB  scores,  duty  position,  training 
history,  etc.),  events  occurring  during  tne  retention 
intervals  (e.g.,  individual  training,  unit  exercises, 
etc.),  and  the  interactions  among  these  variables.  Some  of 
these  variables  are  potentially  important  in  their  own 
right  (i.e.,  independent  of  the  UDA  validation  and  the 
specific  13B  context),  with  possible  implications  for  Army 
selection,  classification,  and  training  decisions.  Others 
are  of  less  general  interest  but  are  nonetheless  of 
importance  to  Field  Artillery  personnel. 

Thus,  this  chapter  has  three  major  sections.  First, 
we  present  analyses  concerned  specifically  with  the  UDA 
validation:  How  accurately  do  UDA  predictions  correspond  to 
actual  performance?  Second,  we  compare  UDA  to  other 
procedures:  Are  UDA  predictions  superior  to  other 
approaches  that  might  be  used  to  predict  proficiency 
levels?  Third,  we  present  analyses  aimed  at  teasing  apart 
the  various  influences  on  retention  performance: 

Controlling  for  as  many  other  potential  influences  as 
possible,  what  is  the  effect  of  Mastery  versus  Proficiency 
training  on  retention?  Does  mode  of  training  (i.e.,  Mastery 
versus  Proficiency)  interact  with  retention  interval  (i.e., 
does  the  effect  change  over  time)?  To  what  extent  do 
individuals'  ASVAB  scores  covary  with  retention 
performance?  To  what  extent  is  performance  affected  by 
other  influences,  such  as  intervening  practice,  previous 
testing,  or  other  background  variables? 


71 


Predicting  Performance:  UDA  Validation 

To  address  the  first  set  of  questions,  we  used  task 
difference  variables  (i.e.,  information  about  differences 
among  the  entire  set  of  tasks)  to  predict  performance  on 
each  13B  task.  The  basic  question  was  whether  the  13B  UDA 
ratings  reliably  distinguished  among  tasks  in  terms  of  the 
Percent  Soldiers  GO  or  the  average  Percent  Steps  GO  for 
each  task  at  selected  retention  intervals. 

The  UDA  ratings  were  generated  using  the  algorithm 
described  in  Chapter  III.  pive  members  of  the  project 
staff  rated  each  task.  Modal  values  were  calculated  for 
each  UDA  question  to  represent  group  consensus,  and  these 
question  values  were  summed  to  produce  a  summary  score  for 
each  task.  The  summary  scores  were  then  converted  to 
predicted  Percent  Soldiers  GO  for  each  task  at  each 
Retention  test,  again  using  the  procedure  described  in 
Chapter  III.  These  predictions  and  the  corresponding 
retention  data  (Percent  Soldiers  GO)  are  shown  in  Table  24 
for  22  13B  tasks.  (As  opposed  to  previous  tables,  the 
tasks  in  Table  24  are  ordered  from  high  to  low  on  the  basis 
of  their  predicted  performance.) 

We  calculated  Pearson  product-moment  correlations 
between  actual  and  predicted  scores  for  both  Percent 
Soldiers  GO  (i.e..  Table  24)  and  Percent  Steps  GO  at  each 
retention  test.  These  correlations  were  as  follows: 

%  SOLDIERS  GO  %  STEPS  GO 

Retention  1  0.91  0.83 

Retention  2  0.61  0.72 

Retention  3  0.71  0.70 

All  of  the  above  correlations  are  statistically  significant 
(p  <  .01).  It  is  important  to  repeat  that  these  13B  tasks 
were  not  used  in  the  development  of  the  UDA  algorithm 
described  in  Chapter  III;  thus,  these  correlations 
represent  a  "true"  test  of  the  validity  of  the  UDA. 

As  shown  in  Table  24,  there  were  differences  between 
actual  and  predicted  scores  across  the  tasks:  as  indicated 
by  the  means  at  the  bottom  of  the  Table,  predicted  scores 
were  about  four  points  lower  than  actual  scores  at 
Retention  1,  about  seven  points  lower  at  Retention  2,  and 
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Table  24 

Predicted  and  Actual  Performance: 
Percent  Soldiers  GO 


RETENTION  1  RETENTION  2  RETENTION  3 


TASK 

ACT. 

UDA 

ACT. 

UDA 

ACT. 

UDA 

Func.  Check  Breech 

95. 

0 

94 

70.8 

88 

79. 

6 

83 

Select  Powder 

100. 

0 

94 

100.0 

88 

99. 

1 

83 

Recover  Coll. 

96. 

6 

89 

57.3 

79 

91. 

2 

70 

Disassemble  M60 

79. 

7 

84 

44.3 

71 

92. 

9 

59 

Emplace  Aim.  Posts 

76. 

4 

84 

52.8 

71 

98. 

2 

59 

Assemble  Breech 

94. 

8 

83 

72.  1 

69 

86. 

6 

57 

Set  Fuze 

100. 

0 

83 

65.9 

69 

— 

— 

57 

Clear  M6G 

90. 

0 

82 

85.4 

67 

99. 

1 

55 

Install  Stakes 

76. 

7 

80 

60.7 

64 

85. 

8 

51 

Disassemble  Breech 

90. 

0 

79 

92. 1 

62 

96. 

4 

49 

Func.  Check  M60 

98. 

3 

78 

95.5 

61 

100. 

0 

47 

Read  TM  -  Breech 

88. 

3 

78 

72.2 

60 

78. 

1 

47 

Establish  Commo 

76. 

7 

75 

61.8 

56 

84. 

1 

42 

Assemble  M60 

83. 

3 

71 

82.0 

50 

87. 

6 

36 

Emplace  Col. 

95. 

0 

65 

49.4 

42 

92. 

9 

27 

Select  Ammo 

63. 

3 

61 

09.2 

37 

30. 

1 

23 

Record  Information 

52. 

6 

55 

57.3 

30 

57. 

5 

17 

Measure  Quadrant 

47. 

5 

54 

63.5 

29 

73. 

4 

16 

Sight  Target 

35. 

6 

51 

18.6 

26 

34. 

2 

13 

Set  Cannon 

59. 

3 

*•9 

50.0 

24 

68. 

8 

12 

Micrometer  Test 

32. 

2 

33 

43.8 

11 

69. 

6 

04 

Boresight  Tele. 

20. 

4 

32 

20.5 

10 

45. 

1 

03 

MEAN 

75.1 

70.6 

60.2 

52.9 

78.6 

40.6 

s.d. 

24.1 

18.1 

24.0 

23.4 

21.1 

24.4 

73 
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38  points  lower  ac  Retention  3.  Note  that  the  correlations 
between  actual  and  predicted  performance  were  maintained 
despite  this  large  difference  at  Retention  3;  this 
indicates  that  the  UDA  continued  to  order  the  tasks 
correctly  despite  the  anomalous  increase  in  actual 
performance  found  at  Retention  3. 

Table  24  also  shows  some  rather  large  differences 
between  actual  and  predicted  scores  for  individual  tasks 
(e.g.,  Emplace  Collimator,  Function  Check  M60).  However, 
in  general  there  is  good  agreement.  It  might  have  been 
expected  that  the  predicted  scores  would  be  lower  than 
actual  scores,  given  the  influences  on  performance 
discussed  in  Chapter  II;  these  influences  (e.g.,  practice. 
Mastery  training)  would  tend  to  inflate  observed  retention 
scores.  It  is  impossible  to  tell  whether  the  differences 
shown  in  Table  24  stem  from  imprecision  in  the  UDA  or  from 
confoundings  in  the  measurement  of  performance;  there  are 
no  consistent  patterns  of  differences  that  could  be 
attributable  to  systemic  deficiencies  in  the  UDA.  On  the 
whole,  these  results  offer  strong  evidence  for  the  validity 
of  the  UDA,  particularly  with  respect  to  its 
generalizabi lity  across  different  Army  occupations. 


Predicting  Performance;  Other  Approaches 

A  second  set  of  analyses  addresses  the  question  of  the 
predictive  power  of  the  UDA  ratings  compared  to  other 
variables  that  could  be  used  to  predict  performance.  As 
previously  discussed  (Rose,  et  al.,  1983),  there  are 
several  alternatives  to  a  purely  analytic  approach  based  on 
estimates  of  task  difficulty.  For  example,  one  could 
collect  empirical  data  similar  to  those  collected  during 
our  Acquisition  Phase.  Information  on  first-trial 
performance  and/or  number  of  trials  to  reach  proficiency 
are  reflections  of  task  difficulty,  and,  if  shown  to  be 
related  to  Retention  performance,  could  be  used  as  the 
basis  for  a  predictive  model.  Likewise,  one  cculd  use 
information  similar  to  our  training  and  practice  data 
(e.g.,  recency  and  frequency  of  individual  training,  number 
of  ARTEP  or  other  live-fire  exercises,  etc.)  as  Retention 
performance  predictors. 

Assuming  equal  predictive  validity  (an  assumption  that 
will  be  evaluated  below),  each  of  these  approaches  has  its 
advantages  and  disadvantages.  For  example,  they  differ  in 
terms  of  the  ease  with  which  relevant  data  could  be 


collected.  UDA  ratings  are  the  easiest  to  collect,  since 
the  rating  process  is  entirely  analytic.  The  only 
requirements  are  for  task-descriptive  information  and  a  set 
of  subject-matter  experts  to  serve  as  raters.  The 
alternative  of  obtaining  data  on  soldiers'  task  experience 
would  require  at  least  access  to  their  individual  Job  Books 
or  individual  soldier  interviews.  The  alternative  of 
collecting  task  performance  data  would  be  the  most 
difficult,  assuming  that  it  is  not  routinely  done.  Such  an 
approach  would  require  development  of  tests,  training  of 
scorers,  setting  up  of  test  facilities,  etc. 

Alternative  approaches  to  predicting  Retention 
performance  also  differ  in  the  generalizabi lity  of  their 
predictions.  It  is  highly  unlikely  that  one  could  use 
Acquisition  performance  data  from  the  tasks  in  one  MOS  to 
predict  retention  performance  on  tasks  in  a  different  MOS. 
Similarly,  use  of  practice  and  training  information  would 
likely  have  limited  applicability  across  MOSs. 

Nevertheless,  we  investigated  two  alternative 
approaches  to  predicting  Retention  performance.  The 
predictor  variables  used  were  (a)  Army  training  (i.e., 
practice)  information,  and  (b)  actual  soldier  performance 
at  Acquisition. 

The  Army  Training  predictor  variable  was  an  average  of 
the  ARTEP/Live  Fire  and  Individual  Training  variables 
collected  from  soldier  self-reports  and  Job  Book 
information  (see  Chapter  II  and  Appendix  D).  The 
ARTEP/Live  Fire  variable  was  scored  as  the  number  of  either 
ARTEP  or  Live  Fire  exercises,  summed  across  Retention 
periods.  For  example,  those  soldiers  attending  Retention  2 
who  reported  at  least  one  ARTEP  or  Live  Fire  exercise  prior 
to  their  test  received  a  "one"  for  the  ARTEP/Live  Fire 
variable;  soldiers  at  Retention  3  who  reported  either 
exercise  prior  to  Retention  2  and  again  prior  to 
Retention  3  were  assigned  a  "two."  Individual  Training  — 
whether  or  not  the  soldier  had  received  training  during  the 
retention  period  — -  was  scored  similarly.  (Using  presence 
versus  absence,  as  opposed  to  the  actual  frequency 
reported,  resulted  in  less  skewed  distributions  for  both 
the  ARTEP/Live  Fire  and  Individual  Training  variables.) 

The  Army  Training  variable  was  an  average  of  the 
ARTEP/Live  Fire  and  Individual  Training  variables.  Each 
task  was  assigned  a  value  by  computing  an  average  score  for 
all  soldiers.  The  resulting  score  was  a  composite  that 
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captured  the  accumulated  training  exercises  and  individual 
training. 

The  Acquisition  Performance  variable  was  the  Percent 
Soldiers  GO  or  Percent  Steps  GO  dependent  measure  (averaged 
across  soldiers)  for  each  of  the  22  tasks  at  first-trial 
Acquisition.  Values  for  the  Acquisition  variable  were 
calculated  for  each  subset  of  soldiers  tested  at  a 
particular  Retention  period.  Thus,  for  example,  the 
Acquisition  data  used  for  predicting  Retention  1  are  not 
based  on  the  same  number  of  soldiers  as  the  Acquisition 
data  used  for  predicting  Retention  2. 

We  considered  three  predictor  variables  for  each  task: 
the  UDA  rating,  the  Army  Training  score,  and  the 
Acquisition  Performance  score.  We  used  step-wise 
regression  analyses  to  examine  the  relationships  between 
the  predictor  variables  and  the  two  criterion  measures.  In 
one  set  of  step-wise  regression  analyses  the  UDA  variable 
was  entered  first,  followed  by  the  Army  Training  variable. 
In  the  second  set  the  UDA  variable  was  entered  first, 
followed  by  the  Acquisition  Performance  variable.  In 
addition  to  generating  zero-order  correlations  between  the 
predictors  and  the  retention  data,  the  step-wise  procedure 
provided  two  other  coefficients  of  interest  —  Beta  weights 
and  Ajr2s. 

Table  25  summarizes  the  results.  With  but  two 
exceptions,  all  three  variables  correlate  significantly 
(p  <  .05)  with  performance  at  all  retention  intervals. 

Based  upon  the  magnitude  of  these  correlations,  either  the 
UDA  score  or  Acquisition  Performance  could  be  used  to 
predict  Retention  performance  with  fair  to  excellent 
accuracy,  depending  upon  the  Retention  period. 

Correlations  are  highest  for  Retention  1,  and  lowest  for 
Retention  2. 

The  Beta  weights  shown  in  Table  25  represent  the 
relative  contributions  of  pairs  of  variables  in  predicting 
Retention  performance.  Th^  Beta  weight  is  not  affected  by 
the  order  in  which  variables  are  entered  into  the 
regression.  The  Beta  weights  thus  answer  the  question 
regarding  which  variable  is  the  "better"  predictor,  UDA  or 
an  alternative. 

The  UDA  rating  is  a  better  predictor  of  both 
performance  measures  at  Retention  1;  it  is  also  the  better 
predictor  for  the  Percent  Steps  GO  dependent  measure  at 
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Table  25 

Zero-Order  Correlations 
and  Multiple  Regression  Results 


Zero-Order  %  Soldiers  %  Steps 

Correlations  GO  GO  v 

%  Soldiers  GO  %  Steps  GO  Beta  Ar2  Beta  Ar 

Retention  1 


UDA 

0.91** 

0.83** 

0.67** 

0.82 

0.58** 

0.69 

Acq.  Perf. 

0.80** 

0.76** 

0.35** 

0.07 

0.40** 

0.10 

UDA 

0.87** 

0.82 

0.79** 

0.69 

Army  Train. 

0.40 

0.39 

0.11 

0.01 

0.13 

0.01 

Retention  2 

UDA 

0.61** 

0.72** 

0.34 

0.38 

0.48* 

0.52 

Acq.  Perf. 

0.64** 

0.69** 

0.41 

0.09 

0.39* 

0.09 

UDA 

0.58* 

0.38 

0.62** 

0.52 

Army  Train. 

0.43* 

0.55* 

0.07 

0.00 

0.16 

0.09 

Retention  3 

UDA 

0.71** 

0.70** 

0.38 

0.50 

0.43* 

0.49 

Acq.  Perf. 

0.75** 

0.71** 

0.50* 

0.14 

0.45* 

0.13 

UDA 

0.65** 

0.50 

0.54** 

0.49 

Army  Train. 

0.44* 

0.59** 

0.10 

0.01 

0.31 

0.07 

*£  <  .05,  two- 
**£  <  .01,  two- 

tailed 

tailed 
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Retention  2.  Acquisition  Performance  is 
predictor  for  both  measures  at  Retention 
better  than  the  Army  Training  variable  at 
testing . 


a  slightly 
3,  but  UDA 
that  same 


better 

is 


The Ar2s  shown  in  Table  25  reflect  the  proportion  of 
criterion  variance  explained  by  each  pair  of  variables. 
This  coefficient  affected  by  the  order  in  which  the 
variables  enter  the  regression;  in  the  present  case,  it 
reflects  the  amount  of  information  the  UDA  predicts, 
including  any  common  variance  between  the  UDA  and 
Acquisition  or  Army  Training  variable. 


An  examination  of  the Ar2s  reveals  that  there  is  a 
substantial  amount  of  overlap  between  the  UDA  and 
Acquisition  or  Army  Training  variables.  For  example, 
consider  the  Percent  Soldiers  GO  variable  at  Retention  1: 
the  UDA  variable  accounts  for  82%  of  the  variance  in  the 
dependent  measure;  once  this  variance  is  "removed,"  the 
Acquisition  Performance  variable  accounts  for  only  an 
additional  7%,  despite  having  a  zero-order  correlation  of 
r  =  0.80  with  Retention  performance;  the  Army  Training 
variable  accounts  for  virtually  no  additional  variance. 

An  important  point  revealed  by  Table  25  is  that  even 
where  Acquisition  Performance  is  the  better  predictor  (as 
determined  by  its  larger  Beta  weight),  the  UDA  accounts  for 
most  of  the  variance  that  would  have  been  predicted  by 
Acquisition  Performance.  In  other  words,  very  little 
information  is  added  by  the  inclusion  of  both  variables. 
Naturally,  these  results  would  have  been  different  if  the 
Acquisition  Performance  variable  had  been  entered  into  the 
regression  before  the  UDA  score;  Acquisition  Performance 
would  have  accounted  for  approximately  64%  of  the  variance 
(estimated  from  the  correlation),  and  the  UDA  would  have 
accounted  for  an  additional  25%  of  the  variance  in  the 
Percent  Soldiers  GO  measure  at  Retention  1.  However,  the 
critical  issue  for  Army  planners  and  trainers  is  whether 
the  remaining  unique  variance  predicted  by  Acquisition 
Performance  is  worth  the  significant  added  cost  of  field 
testing  a  large  group  of  soldiers.  We  think  not. 
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Predicting  Performance:  Other  Influences 

The  emphasis  in  this  chapter  has  been  on  the 
development  and  validation  of  the  UDA.  Toward  that  end  we 
have  reported  on  the  relationships  between  UDA  scores  and 
retention  performance  on  a  variety  of  13B  tasks.  We  have 
also  looked  at  relationships  between  Retention  performance 
on  those  same  tasks  and  other  potentially  useful  predictor 
variables  (i.e.,  Acquisition  Performance,  Army  Training). 
In  this  section  we  examine  the  relationships  between  a 
variety  of  other  independent  variables  and  Retention 
performance.  These  variables  characterize  individual 
differences  among  soldiers  who  participated  in  the 
research.  As  a  consequence,  we  are  interested  in  the 
influence  of  such  variables  on  Retention  performance  for 
each  of  the  22  13B  tasks. 


Two  such  variables  of  primary  interest  are  soldiers' 
ASVAB  scores  and  the  mode  of  training.  Thus,  we  will  look 
in  detail  at  the  ASVAB  Field  Artillery  (FA)  Aptitude  Area 
Composite  score  and  the  Mastery  vs.  Proficiency  training 
variable.  Finally,  we  discuss  in  a  general  fashion  and 
summarize  the  results  of  a  set  of  multiple  regression 
analyses  in  which  we  examined  the  combined  influence  of 
several  individual  difference  variables  on  Retention 
performance.  Toward  that  end  we  conducted  hierarchical 
step-wise  regressions  for  22  tasks,  at  each  Retention 
period,  fo:.  two  or  three  dependent  measures. 


ASVAB  FA  composite  score.  As  discussed  in  Chapter  II, 
a  variable  of  particular  interest  to  this  project  is  a 
soldier's  ASVAB-FA  score.  This  Aptitude  Area  Composite  is 
part  of  the  selection  criteria  for  MOS  13B.  However,  as  we 
mentioned  earlier,  the  soldiers  involved  in  this  project 
had  ASVAB-FA  scores  that  were  statistically 
indistinguishable  from  scores  obtained  from  a  large 
Army-wide  sample  of  soldiers  with  comparable  entry  dates. 


We  considered  using  a  number  of  ASVAB  composite  scores 
(e.g.,  AFQT ,  General  Technical,  etc.).  However,  the  other 
scores  were  correlated  with  FA  and  many  of  them  were 
composites  of  the  same  underlying  ASVAB  subtests.  This 
latter  situation  introduced  some  statistical  problems 
(e.g.,  collincari ty )  that  would  make  interpretation 
difficult.  For  these  reasons,  therefore,  just  the  ASVAB-FA 
score  was  included  in  the  present  analyses. 


79 


& 


,s 


What  is  the  relationship  between  ASVAB-FA  and 
Retention  performance,  independent  of  other  variables? 

There  are  several  ways  of  addressing  this  question.  One  is 
to  examine  the  zero-order  correlations  between  ASVAE'FA  and 
each  performance  measure  at  each  Retention  test.  This 
gives  a  "liberal"  view  of  the  relationships,  since  these 
correlations  include  the  variance  shared  by  ASVAB-FA  and 
other  variables.  More  simply  stated,  high  correlations 
between  ASVAB-FA  and  performance  may  be  due  to  the  fact 
that,  for  example,  high-FA  soldiers  practice  more  than 
low-FA  soldiers.  To  get  a  "pure"  indication  of  the 
relationship  between  ASVAB-FA  and  performance,  we  would 
want  to  eliminate  or  control  for  the  effects  of  practice. 

On  the  other  hand,  if  having  a  high  ASVAB-FA  score  in  some 
sense  causes  soldiers  to  practice  more .  then  the  high 
correlation  would  be  a  true  indication  of  expected 
performance . 

It  would  be  difficult  if  not  impossible  to  untangle 
causality  from  a  pattern  of  correlations  or  by  using  other 
statistical  techniques.  Therefore,  we  will  present  both 
sides  of  the  coin:  we  will  discuss  both  tne  zero-order 
correlations  and  the  most  "conservative"  estimate  of  the 
relationships  —  the  partial  correlations  of  ASVAB-FA  and 
performance,  with  the  effects  of  all  other  variables 
removed . 

We  computed  zero-order  correlations  between  ASVAB-FA 
scores  and  each  performance  measure  (Percent  Soldiers  GO, 
Percent  Steps  GO,  and  time)  for  each  task  at  each  Retention 
test.  There  were  164  such  correlations  (22  tasks  by  three 
measures  by  three  Retention  tests,  minus  tasks  that  were 
not  timed  and  tasks  that  had  no  performance  variance). 
Obtained  values,  presented  in  Tables  20-22,  varied  from 
r  -  -0.34  (Set/Lay  Cannon,  time,  Retention  1)  to  r  =  +0.35 
(Assemble  Breech,  Percent  Soldiers  GO,  Retention  1).  Out 
of  the  164  correlations,  31  were  statistically  significant 
(p  <  .05).  With  few  exceptions,  all  correlations  were  in 
the  expected  direction:  higher  ASVAB-FA  scores  were 
associated  with  better  performance  (producing  negative 
coefficients  for  the  time  performance  measure). 

There  was  no  particular  pattern  discernible  from  the 
significant  correlations;  high  correlations  were  not 
concentrated  at  any  Retention  test  or  for  a  particular 
dependent  measure.  Similarly,  no  particular  type  of  task 
had  more  significant  correlations  than  any  other  type  of 
task.  In  general,  the  best  that  could  be  said  about  the 
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relationship  between  ASVAB-FA  and  Retention  performance  is 
that  the  number  and  direction  of  the  correlations  support  a 
weak  association,  particularly  for  time  data  at 
Retention  1. 

The  more  conservative  test  of  the  relationship  between 
ASVAB-FA  and  performance  is  based  on  the  Beta  weights 
calculated  from  a  multiple  regression  analysis,  where  the 
effects  of  all  other  variables  on  performance  have  been 
removed.  The  "other  variables"  for  these  analyses  will  be 
discussed  more  fully  in  the  final  section  of  this  chapter; 
briefly,  they  included  Mastery  vs.  Proficiency,  Army 
Training,  soldiers*  primary  duty  position,  number  of 
previous  tests,  and  whether  the  soldiers  received  any 
crosstraining  in  crew  positions  other  than  their  primary 
one . 


Again,  164  Beta  weights  were  computed,  corresponding 
to  the  number  of  correlations.  Out  of  these,  32  were 
statistically  significant.  As  opposed  to  the  essentially 
random  pattern  of  significant  zero-order  correlations, 
there  was  some  consistency  here;  two  tasks  had  significant 
Beta  weights  across  all  measures  and  for  each  Retention 
test.  These  tasks  were  Assemble  the  Breech  and  Measure  the 
Quadrant.  Also,  there  was  a  higher  concentration  of 
significant  Beta  weights  for  the  time  measure,  as  opposed 
to  the  two  accuracy  measures.  However,  despite  these 
consistencies,  there  were  still  no  easily  interpretable 
relationships  between  ASVAB-FA  and  Retention  performance. 

In  summary,  one  could  predict  that  high-FA  soldiers 
would  perform  these  tasks  better  than  low-FA  soldiers; 
however,  this  relationship  is  not  strong  enough  to  suggest 
that  a  prediction  of  performance  based  on  a  task's  UDA 
rating  should  be  modified,  given  knowledge  cf  the  soldiers' 
ASVAB-FA  scores. 

Mastery  vs.  Proficiency.  Similar  analyses  were 
conducted  for  the  Mastery-Proficiency  variable.  Recall 
that  each  soldier  received  extra  training  (additional 
trials  following  correct  performance)  on  half  of  the  tasks. 
Thus,  at  each  Retention  test,  half  of  the  soldiers 
performing  each  task  had  received  Mastery  training  and  half 
had  received  Proficiency  training.  The  differences  in 
performance  that  resulted  from  this  differential  treatment 
were  shown  earlier  in  Tables  16-18. 
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Ag?in  we  computed  164  zero-order  correlations  and  Beta 
weights,  one  for  each  task  by  dependent  measure  by 
Retention  test  combination.  For  the  zero-order 
correlations,  only  16  of  the  164  were  statistically 
significant  (p  <  .05).  The  highest  positive  correlation 
was  r  =  .44  (Record  Information,  Percent  Steps  GO, 

Retention  1),  and  the  highest  negative  correlation  was 
r  =  -.35  (Set/Lay  Cannon,  time,  Retention  1).  Clearly, 
there  was  no  systematic  relationship  between  type  of 
training  and  Retention  performance.  This  outcome  is 
similar  to  what  we  found  when  we  compared  mean  levels  of 
Retention  performance. 

Results  were  only  slightly  more  encouraging  when  the 
Beta  weights  were  examined.  Here,  19  of  the  164  Beta 
weights  were  statistically  significant  (p  <  .05).  Five 
tasks  —  Record  Information,  Assemble  Breech,  Select 
Ammunition,  Micrometer  Test,  and  Assemble  M60  —  had  more 
than  one  significant  Beta  weight. 

Given  these  results  and  the  results  presented  in 
Chapter  II,  we  must  conclude  that  the  potential  benefits  of 
additional  training  administered  at  Acquisition  did  not 
accrue.  Naturally,  this  conclusion  is  limited  to  our 
operationalization  of  Mastery  training?  it  is  entirely 
possible  that  more  extensive  or  intensive  training  would 
have  had  greater  positive  effects  on  subsequent  Retention 
performance . 

Other  independent  variables.  The  final  set  of 
analyses  was  an  attempt  to  untangle  the  effects  of  several 
variables  that  potentially  influenced  performance.  The 
strategy  was  to  conduct  large-scale  multiple  regression 
analyses,  including  as  independent  variables  any  and  all 
measures  of  individual  differences  that  we  believed  to  be 
nonredundant  and  reliable.  Dependent  variables  were  again 
Percent  Soldiers  GO,  Percent  Steps  GO,  and  performance  time 
for  each  of  the  22  tasks  at  each  Retention  test. 

The  independent  variables  included  in  these  analyses 

were: 

•  ASVAB  FA  Composite  score 

•  Mastery  vs.  Proficiency 

•  ARTEP/Live  Fire  (see  p.  76) 
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•  Individual  Training  (see  p.  76) 


•  Primary  Duty  Position  — 
Gunner,  and  Cannoneer  #1 
all  other  duty  positions 


Gunner,  Assistant 
received  a  "1," 
received  a  ”0" 


•  Crosstraining  —  based  on  self-report  data, 
whether  or  not  (1  vs.  0)  a  soldier  received 
training  in  duty  positions  other  than  his 
primary  duty  position 


•  Number  of  Times  Tested  During  the  Experiment 

These  variables  were  selected  from  a  larger  set  of 
potential  predictors.  For  example,  we  collected  a  wide 
variety  of  information  regarding  soldiers'  task  experiences 
between  Retention  tests  (see  Appendix  B)  and  extensive 
information  about  the  soldiers  themselves  (e.g.,  rank,  pay 
grade).  As  a  preliminary  step  to  the  present  analyses, 
intercorrelations  among  the  larger  set  of  potential 
predictors  were  calculated,  along  with  descriptive 
information  (mean,  standard  deviation,  skewness,  etc.)  for 
each  variable.  Variables  were  excluded  from  further 
analysis  for  several  reasons.  For  example,  a  variable  was 
excluded  if  it  had  a  high  correlation  with  another 
variable,  or  if  its  distribution  was  highly  skewed,  or 
there  was  little  or  no  variance  in  our  sample  of  soldiers 
on  the  measure,  The  remaining  variables  thus  measured 
different  aspects  of  individual  differences  among  our 
sample  of  soldiers. 


To  reiterate,  these  analyses  were  conducted  with 
several  goals  in  mind.  First,  we  wanted  to  assess  the 
independent  effects  of  these  variables  on  Retention 


being  to  refine  the 
some  cases,  these 
Unit  personnel  when 
For  example,  if  ve 
Retention  performance, 
for  training.  Another 


performance,  with  the  ultimate  goal 
prodictions  for  specific  tasks.  In 
effects  (if  found)  could  be  used  by 
determining  training  requirements, 
found  that  "Crosstraining "  improved 
there  would  be  obvious  consequences 
goal  was  to  assess  the  impact  of  these  variables  when  any 
artifacts  caused  by  our  experimental  treatment  were 
eliminated.  For  example,  our  design  involved  testing  some 
soldiers  two  or  three  times  during  a  six-month  period;  this 
is  not  a  training  strategy  that  would  be  used  in  the  Unit. 
Therefore,  if  this  treatment  influenced  performance,  we 
would  want  to  statistically  "remove"  this  influence  when 
assessing  the  effects  of  other  variables. 
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As  might  be  expected,  the  results  of  these  analyses 
were  quite  complex.  One  way  of  summarizing  them  is  shown 
in  Table  26.  This  table  shows  the  multiple  correlations 
for  the  variables  that  were  significantly  related  to 
performance  for  each  task/measure/Retention  test 
combination.  Only  those  multiple  correlations  that  were 
significantly  greater  than  zero  are  included. 

The  data  indicate  that  some  of  the  variance  in 
Retention  performance  can  be  accounted  for  by  our  set  of 
individual  difference  predictor  variables.  There  are 
significant  multiple  correlations  associated  with  most 
tasks.  For  some  tasks  ■ —  Measure  Quadrant,  Sight  Target, 
Micrometer  Test,  Assemble  the  Breech,  and  Record 
Information  —  the  multiple  correlations  are  significant 
for  practically  all  dependent  measure/Retent ion  test 
combinations.  At  least  half  of  the  tasks  have  significant 
multiple  correlations  at  each  Retention  test  for  the 
Percent  Soldiers  GO  and  time  measures,  and  at  Retention  3 
for  the  Percent  Steps  GO  measure.  In  nearly  70%  of  the 
task/Retent ion  test  combinations,  there  are  significant 
multiple  correlations  with  the  time  measure. 

On  the  other  hand,  the  magnitude  of  these  multiple 
correlations  is  at  best  moderate;  of  the  95  significant 
ones,  roughly  half  (49)  account  for  less  than  10%  of  the 
variance  observed  in  the  performance  measure.  In  fact, 
only  eleven  accounted  for  more  than  20%  of  the  variance;  at 
best  (Assemble  Breech,  Retention  2,  time,  and  Set/Lay 
Cannon,  Retention  1,  time),  the  individual  difference 
variables  accounted  for  32%  of  the  variance  (multiple 
correlation  =  .57). 


To  conclude  this  discussion.  Table  27  shows  the 
specific  variables  that  were  significantly  related  to 
performance  in  each  cell.  As  can  be  seen,  many  of  the 
effects  in  specific  cells  are  complex;  there  are  two-way 
and  three-way  interactions,  negative  relationships,  and 
suppressor  effects  (i.e.,  variables  that  enhance  the 
relationship  between  another  variable  and  performance, 
while  not  itself  being  related  to  performance). 

The  gist  of  thene  analyses  is  that  there  were  many 
influences  on  Retention  performance  other  than  the 
particular  variables  of  interest  to  this  project  (i.e., 
Proficiency-Mastery,  ASVAB-FA).  However,  when  one 
considers  the  range  of  variables  included  in  these 
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Table  26 

Multiple  Regression  Results:  Significant  Multiple  Correlations 
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Not*:  significance  is  at  (he  .05  level 


Table  26  (continued) 

Multiple  Regression  Results:  Significant  Multiple  Correlations 
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Note:  significance  is  at  the  .05  level 


Table  27 

Multiple  Regression  Results:  Significant  Predictors 
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Table  27  (continued) 

Multiple  Regression  Results:  Significant  Predictors 
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analyses,  the  result  (i.e.,  that  only  a  moderate  amount  of 
performance  variance  was  accounted  for)  is  rather 
surprising.  Especially  when  compared  to  the  results 
reported  above  concerning  the  relationship  between  the  UDA 
ratings  and  Retention  performance,  the  implication  is  that 
by  far  the  biggest  determinant  of  Retention  performance  is 
the  characteristics  of  a  task,  especially  those 
characteristics  measured  by  the  UDA.  This  means  that  the 
UDA  alone,  without  embellishments  and  qualifications 
concerning  soldiers'  abilities,  experience,  etc.,  could 
provide  good  predictions  of  Retention  performance. 
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APPENDIX  A 


HANDS-ON  TESTS  FOR  TASKS 
SELECTED  FOR  RETENTION  EXPERIMENT 


PREPARE  A  POSITION  TO  RECEIVE/ EMPLACE  A  HOWITZER 


Equipment  Required  To  Set  Up  Station  and  Conduct  Test. 

2  Gun  guide  stakes 
2  Pantel  marking  stakes 
2  M2  compasses 

2  DR3,  with  field  wire 

3  TA-312/PT  field  telephones,  with  batteries 

1  SB-16  switchboard 

2  Screwdrivers,  flattip 

1  Aiming  circle 
Paper  and  pencil 

2  Tent  stakes 
2  Hammers 


Procedures  To  Set  Up  Station 

1.  Orient  aiming  circle  on  open  area.  Determine  a  reasonable  azimuth  of  fire 
to  tell  soldiers  in  instruction. 

2.  Lay  out  following  equipment  near  aiming  circle: 

a.  Pantel  marking  stakes 

b.  Gun  guide  stakes 

c.  Hammers 

d.  TA-312/PT  field  telephones,  with  batteries 

e.  Screwdrivers 

f.  M2  compasses 

g.  DR8  with  field  wire 

h.  Paper  and  pencil 

3.  Set  up  switchboard  about  5  meters  from  aiming  circle. 


Procedures  To  Be  Performed  Before  Testing  Each  Soldier 


1. 

Return  equipment  used  by  soldier  to 

aiming  circle. 

2. 

Offset  TA-312/PT: 

a. 

Remove  batteries. 

b. 

Set  buzzer  control  knob  to  LOW. 

c* 

Place  I NT- EXT  to  EXT. 

d. 

Place  circuit  selector  switch  to 

CB. 

A- 1 


Soldier : 


Scorer: 

Date: 


SCORESHEET 

PREPARE  A  POSITION  TO  RECEIVE/ EMPLACE  A  HOWITZER 


INSTRUCTIONS  TO  SOLDIER:  During  this  test  you  will  act  as  a  gun  guide 
preparing  a  position  to  receive  a  howitzer.  The  equipment  you  will  need  is  by 
the  aiming  circle.  You  may  record  information  on  the  tablet  (hand  soldier  the 

paper  and  pencil).  You  are  gun  _ .  Your  azimuth  of  fire  is 

_ .  Emplace  the  pantel  marking  stake  (designate  area  about  30  meters 

away  at  an  oblique  angle.) 

TALK  THROUGH 


PERFORMANCE  MEASURES  GO  NO-GO  COMMENTS 


Install  Stakes 

1.  Emplaced  pantel  marking  stake  in  area 
designated.  (Must  not  move  stake  while 
alining  gun  guide  stake.) 


2.  Sighted  through  M2  compass  to  determine  the 
azimuth  of  fire  from  pantel  marking  stake. 


3.  Emplaced  gun  guide  stake  along  azimuth  of 
fire,  at  least  three  paces  from  pantel 
marking  stake. 


4.  Final  alinement  between  gun  guide  stake  and 
pantel  marking  stake  +  20  mils  of  directed 
azimuth. 


Establish  Communication 

5.  Installed  jack  in  switchboard  position  number 
for  howitzer. 


6.  Installed  field  telephone  in  area  near  pantel 
stake  clear  of  area  where  howitzer  would  be 
positioned  (from  gun  guide  stake,  left  of 
pantel  stake) . 


7.  Tied  wire  to  tent  stake  at  telephone. 

8.  Connected  wire  to  binding  posts  of  tele 
phone. 


9.  Installed  batteries  (one  +  up:  one  +  down). 


10.  Turned  selector  switch  to  LB. 


A-2 


SCORESHEET  (Cont'd.) 

PREPARE  A  POSITION  TO  RECEIVE/ EMPLACE  A  HOWITZER 


PERFORMANCE  MEASURES  GO  NO-GO  COMMENTS 

11.  Turned  INT-EXT  switch  to  INT.  _  _ _ 

12.  Performed  ring  test  by  holding  receiver  in 

cradle  while  turning  crank.  _  _ 

13.  Conducted  commo  check  with  aiming  circle 

telephone.  _ 

Recorded  Information 


NOTE  TO  SCORER:  Scorer  Assistant  should  tell 
soldier  the  deflection  whenever  he  asks  for  it. 
If  he  does  not  ask,  have  the  Scorer  Assistant 
give  the  deflection  as  part  of  the  commo  check. 

14.  Recorded  deflection  from  aiming  circle  to 
Pantel  marking  stake. 

15.  Paced  off  distance  from  aiming  circle  to 
Pantel  marking  3take  and  recorded  it. 


TIME  Initial  Retest 

1.  Minutes  to  install  stakes:  First  Test 

Repetition  2  _  _ 

Repetition  3  _ _ 

2.  Minutes  to  establish  communication:  First  Test 

Repetition  2  _____  _ 

Repetition  3  _  _ 

PM  NO-GO  Repetition  1:  Install  Stakes  (1-4) 

Est.  Commo  (5-13)  _ _ 

Rec.  Info  (14,  15)  _ _ 

PM  NO-GO  Repetition  3:  Install  Stakes  (1-4)  _ 

Est.  Commo  (5-13)  _ _ _ 

Rec.  Info  (14.,  15) 


EMPLACE/ RECOVER  AIMING  POSTS 
EMPLACE/ RECOVER  COLLIMATOR 


Equipment  Required  To  Set  Up  Station  and  Conduct  Test 
M198  howitzer 

M137  panoramic  telescope  (mounted) 

Set  of  aiming  posts  (four  sections  -  complete) 

Night-lighting  device  for  aiming  posts 
Scorer  assistant  to  act  as  gunner 
M1A1  collimator 

Procedures  To  Set  Up  Station 

1.  Position  howitzer  in  location  that  has  at  least  100  meters  of  level 
terrain. 

2.  Orient  howitzer  on  direction  of  fire  with  tube  in  center  of  traverse  at 
load  elevation. 

3.  Orient  pantel  on  direction  of  area  where  aiming  posts  will  be  emplaced. 

Procedures  To  Be  Performed  Prior  To  Testing  Each  Soldier 

1.  Place  one  set  of  aiming  posts  (four  short  sections)  to  left  side  of 
howitzer. 

2.  Place  night-lighting  devices  next  to  aiming  posts. 

3.  Turn  azimuth  adjusting  knob  on  collimator  fully  left  or  fully  right. 

During  the  test,  offset  the  knob  before  the  soldier  recovers  the 
collimator. 

4.  Place  collimator  to  left  side  of  howitzer  in  stowed  position. 

Scoring  Procedures 

Aiming  Post 

1.  Score  the  aiming  post  test  from  the  panoramic  telescope.  Give  the  soldier 
signals  for  the  adjustment. 

2.  For  PM  3  and  4,  count  the  steps  the  soldier  takes  to  the  emplacement 
point. 

3.  If  the  soldier  gets  a  NO-GO  on  the  aiming  post  test,  tell  him  what  he  did 
wrong  as  soon  as  he  finishes.  But  give  the  collimator  test  before  you 
retest  him  on  the  aiming  post.  (This  is  to  give  him  time  to  recover  his 
breath. ) 

4.  Score  the  collimator  test  by  the  soldier  while  the  scorer  assistant  acts 
as  gunner. 

Collimator 

5.  For  PM  10,  the  soldier  may  not  need  to  center  the  knob.  If  he  can  aline 
the  collimator  without  centering  the  knob,  enter  NA  for  the  PM. 

If  the  collimator  is  alined  without  any  adjustment,  have  the  gunner  aline 
five  mils  left  or  right. 


6. 


Scorer: 


Soldier: 


Date: 


SCORESHEET 

EMPLACE/RECOVER  AIMING  POSTS 


INSTRUCTIONS  TO  SOLDIERS:  For  this  test  you  must  emplace  and  recover  aiming 
posts.  Assume  we  will  occupy  this  position  for  next  24  hours.  You  will  have 
two  minutes  to  amplacc  the  aiming  posts.  I  will  act  as  Gunner.  Begin. 

PERFORMANCE  MEASURES  GO  NO-GO  COMMENTS 

1.  Prepared  two  short  posts  for  far  post.  _ _  _  _ _ 

2.  Attached  night-lighting  device  to  top  of 

each  aiming  post.  _  _____  _ 

3.  Positioned  near  post  (short  section)  approx¬ 
imately  SO  meters  from  howitzer  (33-36 

steps) .  _  _  _ 

4.  Positioned  far  post  (long  section)  same 
distance  (+  3  steps)  from  short  post  as  short 

post  was  from  howitzer.  _ _ _ 

5.  Turned  night-lighting  device  on  far  post 

toward  howitzer.  _  _  _ 

6.  Emplaced  far  post  _  _  _ _ 

7.  Emplaced  near  post  at  direction  of  gunner.  _  ______  _ 

8.  Turned  night-light ing  device  on  near  post 

toward  howitzer.  _  _  _ 

9.  Completed  emplacement  within  two  minutes.  _  _  __ _ 

NOTE  TO  SCORER:  After  soldier  completes  amplacing 
near  post,  give  verbal  command  RECOVER  AIMING  POSTS. 

10.  Pulled  posts  from  ground.  _  _ _  _ 

11.  Disassembled  far  posts.  _  _ _  _ 

12.  Removed  night-lighting  devices.  _  _  _ 

13.  Replaced  posts  and  devices  in  stowed  posi¬ 
tion.  _  _  _ _ 


TIME  Initial  Retest 

Minutes  to  emplace  aiming  posts:  First  Test  _  _ 

Repetition  2  _  _ 

Repetition  3  _ _  _ 

PM  NO-GO  Repetition  2  _ _ 

PM  NO-GO  Repetition  3  _ 


A- 5 


Soldier: 


Scorer:  _ 

Date:  _ _ 

S CORESHEET 
EMPLACE/ RECOVER  COLLIMATOR 


INSTRUCTIONS  TO  SOLDIER:  For  this  test  you  must  emplace  and  recover  the 
collimator..  The  assistant  will  act  as  the  Gunner.  First  emplace  the 
collimator.  You  have  two  minutes.  Begin. 


PERFORMANCE  MEASURES  GO  NO-GO  COMMENTS 

Emplace  Collimator 

1.  Set  up  4  to  15  meters  from  pantel  and  more 

than  45*  from  muzzle  end.  _  _  _ _ 

2.  Unlatched  leg  holding  straps  and  lowered 

legs.  (Does  not  have  to  extend  legs.)  _  _  _ _ 

Removed  cover.  _  _ 

4 ..  Placed  cover  under  tripod-dosed  end  toward 

howitzer.  _  _ 

5 .  Loosened  azimuth  clamping  knob.  _  _  _ 

A.  Loosened  cross-level  damping  knob.  _  _  _ 

.  Loosened  elevation  clamping  knob.  _  _  _ 

$  Placed  collimator  in  horizontal  position.  _  _  _ 

).  Opened  lens  cover  before  gunner  started 

adjustment.  _  _ _ 

'.entered  azimuth  adjustment  knob  (if  needed).  _  _ _ _ 

.  lined  front  and  rear  sights  of  collimator 
on  pauorastlc  telescope  (rough  lay) .  _  _ _  _ 

12.  Tightened  azimuth  clamping  knob.  _  _  _ 

13.  Tightened  elevation  clamping  knob.  _  _  _ _ 

14.  Leveled  cross- level  bubble  after  dosing 

azimuth  and  elevation  clamping  knobs.  _  _  _ _ 

15.  Tightened  cross- level  clamping  knob.  _  _  _ 

16.  Alined  collimator  by  rotating  azlnuth 

adjusting  knob  following  gunner's  signals.  _  _  _ 

17.  Emplaced  collimator  within  two  minutes.  _  _  _ 


A-6 


SCORESHEET  (Cont'd.) 
EMPLACE/ RECOVER  COLLIMATOR 


PERFORMANCE  MEASURES  GO  NO-GO  COMMENTS 

INSTRUCTIONS  TO  SOLDIER:  Now  recover  the  colli¬ 
mator.  You  have  one  minute. 

18.  Loosened  elevation  clamping  knob.  _  _  _ 

19.  Placed  collimator  in  vertical  position.  _  _  _ 

20.  Tightened  elevation  clamping  knob.  _  _ _  _ 

21.  Fastened  snap.  _  _  _ 

22.  Placed  cover  on  collimator  and  closed  snaps.  _  _  _ _ 

23.  Folded  legs.  _  _  _____ _ 

24.  Resnapped  holding  straps.  _ _ _ _ _ _____ 

25.  Recovered  collimator  within  one  minute.  _____  ______  ___________ 

NOTE  TO  SCORER:  Returning  collimator  to  the 
howitzer  is  not  timed. 

TIME  Initial  Retest 

Minutes  to  emplace  collimator:  First  Test  _  _ 

Repetition  2  _  _ _ 

Repetition  3  _  _ 

PM  NO-GO  Repetition  2  _ 

PM  NO-GO  Repetition  3  _ 
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PERFORM  OPERATOR  MAINTENANCE  ON  THE  M60  MACHINEGUN 
(ASSEMBLY/DISASSEMBLY) 


Equipment  Required  To  Set  Up  Station  and  Conduct  Test 

M60  machinegun  (with  bolt) 

Small  punch  or  tool  in  spare  barrel  bag 
Tarpulin 


Procedures  To  Set  Up  Station 

1.  Place  M60  on  tarpaulin  with  tripod  legs  open. 

2.  Place  small  punch  by  M60. 

Procedures  To  Be  Performed  Before  Testing  Each  Soldier 

1.  Insure  M60  is  returned  to  operational  condition  if  it  does  not  pass 
function  check. 

2.  Set  safety  on  SAFE. 


Procedures  To  Conduct  the  Test 


1.  If  the  soldier  does  not  remove  the  components  required  fcr  the  eight 
groups,  mark  NO-GO  for  the  incomplete  group  and  tell  him  to  remove  the 
components . 

2.  If  the  soldier  gets  a  NO-GO  on  Clear  or  Disassemble,  give  the  remediation 
after  the  weapon  is  assembled. 

3.  Treat  this  test  like  three  tasks:  Clear  and  Disassemble;  Assemble;  and 
Function  Check.  If  the  soldier  gets  a  NO-GO  on  one  part  but  not  the 
others,  he  should  repeat  only  the  major  segment  where  he  made  the  error. 

If  necessary,  you  must  assemble  or  disassemble  the  M60  to  get  it  ready  for 
the  soldier. 


Scorer: 


Soldier: 


Date:  _ 

SCORESHEET 

PERFORM  OPERATOR  MAINTENANCE  ON  THE  M60  MACHINEGUN 
(ASSEMBLY/DISASSEMBLY) 


INSTRUCTIONS  TO  SOLDIER:  During  this  test  you  will  disassemble  and  assemble 
the  M60  raachinegun.  First,  disassemble  the  machinegun  into  its  eight  major 
groups.  You  will  have  four  minutes.  Begin. 


PERFORMANCE  MEASURES  GO  NO-GO  COMMENTS 

Clear 

1.  Raised  cover.  _  _ _ 

2.  Placed  safety  to  FIRE  and  cocked  weapon.  _  _  _ 

3.  Visually  checked  chamber.  _  _  _ 

4.  Let  bolt  slide  forward  while  pulling  trigger.  _  _  _ 

NOTE  TO  SCORER:  Do  not  let  soldier  start  dis¬ 
assembly  with  weapon  cocked. 

5.  Placed  safety  to  SAFE.  _ A/ft _  _ 

Disassemble 

6.  Removed  the  stock  group.  _ _  _  _ _ 

7.  Removed  buffer  and  operating  rod  group. 

(Must  separate:  operating  rod  and  bolt; 

spring  and  spring  guide.)  _  _  _ 

8.  Removed  trigger  mechanism  group.  _  _  _ 

9.  Removed  barrel  group.  _ __ _ _ _ 

10.  Removed  forearm  assembly.  _ _ _ 

11.  Removed  cover,  tray  and  hanger  group.  (Must 

remove  cover  spring.)  _ _  _  _ 

12.  Completed  disassembly  within  four  minutes.  _  _  _ 


TIME 

Minutes  to  disassemble  M60  (PM  1-11) :  First  Test 


Repetition  2 
Repetition  3 


PM  NO-GO  Repetition  2 
PM  NO-'GO  Repetition  3 


Initial 


Retest 
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SCORESHEET  (Cont’d.) 


PERFORM  OPERATOR  MAINTENANCE  ON  THE  M60  MACHINEGUN 
(ASSEMBLY/DISASSEMBLY) 


INSTRUCTIONS  TO  SOLDIER:  Now  assemble  the  M60.  You  have  four  minutes. 

PERFORMANCE  MEASURES  GO _  NO-GO  COMMENTS 

Assemble 

1.  Replaced  cover,  tray,  and  hanger  group. 

(Hinge  pin  from  right  side  and  latch  from 

the  left.)  _  _  _ _ 

2.  Replaced  barrel  group.  _  _____  _ 

3.  Replaced  trigger  mechanism  group.  _  _____  _ 

4.  Replaced  bolt  assembly.  _  _  _ 

5.  Replaced  operating  rod  group.  _  _  _ 

6.  Replaced  buffer  assembly.  _  _ 

7.  Replaced  stock  group.  _  _ _  _ 

8.  Completed  assembly  within  four  minutes.  _  _  _ 

Function  Check 

NOTE  TO  SCORER:  If  soldier  does  not  do  function 
check  on  his  own,  tell  him  to  conduct  one. 

9.  Placed  safety  on  FIRE  and  cocked  weapon.  _  _ 

10.  Put  safety  on  SAFE  and  pulled  trigger  (should 

not  fire.)  _  _  _ _ 

11.  Closed  cover  before  riding  bolt  forward.  _  _  _ 

12.  Placed  safety  on  FIRE.  _  _  _ 

13.  Let  bolt  slide  forward  while  pulling  trigger.  _  _  _ 


TIME  Initial  Retest 

Minutes  to  assemble  M60  (PM  1-7) :  First  Test  _  _ _ 

Repetition  2  _  _ 

Repetition  3  _ _  _ _ 

PM  NO-GO  Repetition  2  _ 

PM  NO-GO  Repetition  3  _ _ 
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PREPARE  SEPARATE  LOADING  AMMUNITION  FOR  FIRING 


Equipment  Required  To  Set  Up  Station 

Drawings  of  three  projectiles  (w/color  and  nomenclature  but  without 
type) :  HE,  WP,  Illumination 

Drawings  in  color  of  fuzes:  M557  PD,  M564  MTSQ,  M565  MT,  M514  VT 

Drawings  in  color  of  white  bag  and  green  bag  powder  charges 

105  HE  projectile,  inert 

M563  fuze,  mechanical  time,  inert 

M17  fuze  wrench 

Record  of  fire  forms  (DA  Form  4513) 

Easel 

Field  table 


Procedures  To  Set  Up  Station 

1.  Place  drawings  on  easel. 

2.  Place  projectile,  fuze,  and  wrench  on  field  table. 


Procedure  To  Be  Performed  Prior  To  Testing  Each  Soldier 
1.  Set  fuze  to  0. 


Procedures  To  Conduct  Test 


1.  Give  the  fire  commands  at  real  time  speed. 

2.  If  the  soldier  requests  SAY  AGAIN  for  any  element  of  the  command,  announce 
it  again.  (Do  not  score  NO-  0.) 

3.  If  the  soldier  changes  a  response  before  you  give  the  next  command,  score 
the  last  response. 
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Soldier: 


Scorer:  _ 

Date:  _ 

SCORESHEET 

PREPARE  SEPARATE  LOADING  AMMUNITION  FOR  FIRING 


INSTRUCTIONS  TO  SOLDIER:  During  this  test  you  will  do  some  of  the  steps 
associated  with  preparing  separate  loading  ammunition  for  firing.  I  will 
announce  fire  commands  and  you  must  select  the  projectile  and  fuze  that  are 
required  by  each  command.  You  should  fill  out  the  Record  of  Fire  form. 

PERFORMANCE  MEASURES  GO  NO-GO  COMMENTS 

BATTERY  ADJUST,  SHELL-WP  LOT-XY,  CHARGE-5  GREEN 
BAG,  FUZE-QUICK 

1.  Selected  WP  (light  green  with  red)  round.  _  _ _ 

2.  Selected  M557  fuze.  _ 

BATTERY  ADJUST,  SHELL-HE  LOT-XY,  CKARGE-3  GREEN 
BAG,  FUZE  TIME,  TIME  22.6 

3.  Selected  HE  (OD  green  with  yellow)  round.  _ _ 

4.  Selected  M564  fuze.  _ 

5.  Set  22.6  on  fuze.  _  _ 

BATTERY  ADJUST,  SHELL-ILLUMINATION,  LOT-XY, 

CHARGE-5  GREEN  BAG,  FUZE  TIME,  TIME  20.9 

6.  Selected  Illumination  (OD  green  with  white) 
round . 

7.  Selected  M565  fuze.  _ 

BATTERY  ADJUST,  SHELL-HE,  LOT-XY,  CHARGE-4 
GREEN  BAG,  FUZE  VICTOR  TANGO,  TIME  21.0 

8.  Selected  HE  (OD  green  with  yellow)  round.  _ 

9.  Selected  M514  fuze.  _ 

SELECT  THE  POWDER 

10.  Selected  green  bag  powder.  _ 

11.  Selected  charge  1-4.  _ 

BATTERY  ADJUST,  SHELL-WP,  LOT-XY,  CHARGE-5, 

GREEN  BAG,  FUZE  TIME,  TIME  38.9 

12.  Selected  WP  (light  green  with  red)  round.  _  _ 

13.  Selected  M564  fuze.  _ 

14.  Set  39.9  on  fuze. 


PM  NO-GO  Repetition  2 
PM  NO-GO  Repetition  3 
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PERFORM  GUNNER'S  QUADRANT  MICROMETER  TEST/ 
BORESIGHT  DIRECT  FIRE  TELESCOPE  USING  DAP 


Equipment  Required  To  Set  Up  and  Conduct  Test 
M198  howitzer 

14138  direct  fire  telescope 
M1A2  gunner's  quadrant 
Bores ight  string  and  tape 
Screwdriver,  flat tip 


Procedures  To  Set  Up  Station 

1.  Aline  cannon  tube  on  a  well-defined  distant  aiming  point  (DAP)  approx¬ 
imately  1300  meters  from  howitzer. 

2.  Attach  boresight  strings. 

3.  Mount  M138  direct  fire  telescope. 

4.  Conduct  end  for  end  and  micrometer  test  of  gunner's  quadrant.  If  quadrant 
is  not  accurate,  provide  a  replacement. 


Procedures  To  Be  Performed  Before  Testing  Each  Soldier 

1.  Turn  the  elevation  screw. 

2.  Off-set  the  cross-level  bubble  slightly. 


Procedures  To  Conduct  Test 


1.  Conduct  the  micrometer  test  before  the  boresight  test. 

2.  If  the  soldier  centers  the  cross-level  bubble  when  he  performs  the  micro¬ 
meter  test,  tell  him  to  offset  the  bubble  before  he  boresights. 

3.  If  the  soldier  makes  an  error  on  the  micrometer  test,  conduct  remedial 
training  on  the  micrometer  test  before  you  give  the  boresight  test. 
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Scorer:  _ _  Soldier:  _ 

Date:  _ 

SCORESHEET 

PERFORM  GUNNER'S  QUADRANT  MICROMETER  TEST 

INSTRUCTIONS  TO  SOLDIER:  For  this  test  you  must  conduct  a  micrometer  test  on 
the  gunner's  quadrant.  You  have  one  minute.  Begin. 

TALK  THROUGH  _ _ 

PERFORMANCE  MEASURES  GO  NO-GO  COMMENTS 

1.  Set  index  arm  to  +10  or  zero.  _  _  _ 

2.  Set  micrometer  to  zero  or  +10  (opposite  of 

index  arm  setting).  _____  _  _ _ 

3.  Placed  gunner's  quadrant  on  quadrant  seats, 

pointed  toward  muzzle.  _  _  _ _ 

A.  Centered  bubble  on  gunner's  quadrant  by 

depressing  or  elevating  tube.  _  _  _ 

5.  Reversed  settings  on  index  arm  and  micro¬ 
meter.  _  _  _ 

6.  Placed  gunner's  quadrant  on  quadrant  seat3, 

pointed  toward  muzzle.  _  _  _ 

7.  Checked  bubble  on  gunner's  quadrant  without 

moving  tube.  _  _  _ 

8.  Reported  status  of  micrometer.  _  _  _ 

9.  Completed  micrometer  test  within  one  minute.  _  _ _  _ _ 


TIME 


Initial  Retest 


Seconds  to  conduct  micrometer  test:  First  Test 

Repetition  2 
Repetition  3 


PM  NO-GO  Repetition  2 
PM  NO-GO  Repetition  3 
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Scorer:  _  Soldier:  __ 

Date:  _ _ 

SCORESHEET 

BORESICHT  DIRECT  FIRE  TELESCOPE  USING 
DISTANT  AIMING  POINT  METHOD 


INSTRUCTIONS  TO  SOLDIER:  For  this  test  you  must  boresight  the  direct  fire 

telescope  using  the  distant  aiming  point.  The  DAP  is _ _ . 

The  boresight  strings  have  been  mounted  and  the  tube  is  alined  on  the  DAP. 
You  ha"e  three  minutes.  Begin. 

TALK  THROUGH  _ 

PERFORMANCE  MEASURES  GO  NO-GO  COMMENTS 


NOTE  TO  SCORER:  Soldier  may  take  reading  off 

elevation  counter  for  his  initial  reading.  He 

must  confirm  or  refine  the  reading  with  the 

quadrant  before  subtracting  2.3. 

1.  Measured  the  elevation  to  the  DAP  by  placing 
gunner's  quadrant  on  quadrant  seats  and 
leveling  longitudinal  bubble  of  gunner's 
quadrant  (must  not  move  tube). 

2.  Subtracted  2.3  mils  from  elevation  measured 
with  gunner's  quadrant. 

3.  Placed  revised  setting  on  gunner’s  quadrant. 

4«  Depressed  tube  until  longitudinal  bubble  of 
gunner's  quadrant  was  centered. 

5.  Centered  cross-level  bubble,  if  required. 

6.  Checked  gunner's  quadrant  after  centering 
cross-level  bubble  (if  sight  was  adjusted 
in  PM5)  . 

7.  Alined  the  telescope  with  the  gun  tube  by 
turning  the  elevation  adjusting  screw  while 
sighting-  through  direct  fire  telescope. 

8.  Sight  picture  had  0-mil  line  alined  on  the 

DAP. 

9.  Completed  boresight  within  three  minutes. 


TIME  Initial  Retest 

Minutes  to  boresight:  First  Test  _ 

Repetition  2  _  _ 

Repetition  3  _  _ 

PM  NO-GO  Repetition  2  _ _ 

PM  NO-GO  Repetition  3  _ _ _ 
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MEASURE  THE  QUADRANT  WITH  THE  RANGE  QUADRANT 
SET/LAY  THE  CANNON  FOR  QUADRANT  WITH  RANGE  QUADRANT  (M198) 
SIGHT  ON  A  TARGET  WITH  DIRECT  FIRE  TELESCOPE 


Equipment  Required  To  Set  Up  Station  and  Conduct  Test 
M198  howitzer 

M138  direct  fire  telescope 


Procedures  To  Set  Up  Station 

1.  Take  tube  out  of  travel  lock  and  change  elevation  of  gun. 

2.  Insure  elevation  correction  counter  reads  zero  (00). 

3.  Select  or  position  a  target  at  range  of  600-900  meters. 

Procedures  Performed  Before  Testing  Each  Soldier 

Measure  Quadrant 

1.  Set  cannon  at  an  elevation  different  from  previous  test  (if  any). 

2.  Level  all  bubbles  without  moving  the  tube. 

3.  Record  the  quadrant  from  the  elevation  counter. 

4.  Without  moving  the  tube,  change  number  in  elevation  counter  and  off- 
center  the  longitudinal  and  cross-level  bubbles. 

Set/Lay  for  Quadrant 

5.  Off-center  the  cross-level  bubble. 

Sight  on  Target 

6.  No  change  required. 

Procedures  To  Conduct  and  Score  Test 

1.  Give  the  measure  quadrant  test  first,  followed  by  set  quadrant,  then  sight 
on  target. 

2.  For  the  measure  quadrant  test,  you  do  not  need  to  check  whether  the 
bubbles  are  centered,  just  whether  the  soldier  turned  the  knobs. 

3.  For  the  set  quadrant  test  you  do  need  to  check  the  longitudinal  and 
cross- level  bubbles.  Each  bubble  must  be  within  or  on  the  scribe  marks. 

4.  For  the  sight  on  target  test,  alternate  700  and  900  meters  as  the  range 
element.  These  two  ranges  will  require  the  soldier  to  determine  the 
midpoint  of  listed  mil  conversions. 
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Scorer:  Soldier: 


Date: 


S CORESHEET 

MEASURE  QUADRANT  WITH  THE  RANGE  QUADRANT  (M198) 

INSTRUCTIONS  TO  SOLDIER:  At  this  station  you  will  act  as  the  Assistant  Gunner 
and  measure  quadrant  using  the  range  quadrant.  You  must  measure  and  announce 
the  quadrant  within  15  seconds.  Ready.  Measure  the  quadrant. 


QUADRANTS 
Initial  Retest 

First  _ _ 

Second  _  _ 

Third 


PERFORMANCE  MEASURES  GO  NO-GO  COMMENTS 


1.  Centered  the  longitudinal  and  cross-level 
bubbles  without  moving  the  tube. 

2.  Announced  the  quadrant  within  1  mil  of 
recorded  quadrant. 

3.  Announced  quadrant  within  15  seconds  of 
command . 


TIME 


Initial  Retest 


Seconds  to  measure  quadrant:  First  Test 

Repetition  2 
Repetition  3 


PM  NO-GO  Repetition  2 
PM  NO-GO  Repetition  3 
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Soldier : 


Scorer: 

Date: 


S CORESHEET 

SET/LAY  CANNON  FOR  QUADRANT  WITH  RANGE  QUADRANT  (M198) 


INSTRUCTIONS  TO  SOLDIER:  On  this  test  you  will  act  as  the  Assistant  Gunner. 
You  will  set  the  cannon  for  quadrant  using  the  range  quadrant.  You  must  set 
off  the  quadrant  within  30  seconds.  Ready. 

QUADRANT  _  (Give  soldier  quadrant  from  30-50  mils  from  previous 

setting) 


PERFORMANCE  MEASURES  GO  NO-GO  COMMENTS 


1.  Repeated  command  QUADRANT 


2.  Set  announced  quadrant  on  the  elevation 
counter  before  centering  bubbles. 


3.  Centered  the  longitudinal  bubble  by  elevating 
or  depressing  the  cannon  tube. 


4.  Centered  the  cross- level  bubble  using  the 
cross  leveling  knob  after  centering  the 
longitudinal  bubble. 


5.  Announced  SET, 


6.  Set  quadrant  within  30  seconds. 


TIME 


Initial  Retest 


Time  to  set  cannon  for  quadrant:  First  Test 

Repetition  2 
Repetition  3 


PM  NO-GO  Repetition  2 
PM  NO-GO  Repetition  3 
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*  Jt  W  -ff '  K  *  A  ^  .  ^  _  _ 


Scorer: 


Soldier: 


Dote: 


SCORESHEEET 

SIGHT  ON  A  TARGET  WITH  THE  DIRECT  FIRE  TELESCOPE  (M198) 


INSTRUCTIONS  TO  SOLDIER:  Ac  this  station  you  will  act  as  the  Assistant  Gunner 
and  antes*  *  direct  fire  target.  You  have  IS  seconds  to  perform  this  task 
after  the  initial  fire  command  is  given.  Ready. 

NOTE  TO  SCORER:  Give  fire  command:  "TARGET  TRUCK  (RIGHT/LEFT)  FRONT,  SHELL, 
HE,  CHARGE-7  WB,  FUZE  QUICK,  RANGE _  METERS,  FIRE  AT  WILL." 


PERFORMANCE  MEASURES  GO  NO-GO  COMMENTS 


1.  Repeated  announced  range  from  the  fire 
command. 


2.  Elevated  or  depr eased  the  tube  until  the 
mil  line  corresponding  to  the  range  to 
target  was  on  center  maas  of  target. 

3.  Repeatedly  announced  SET  until  command  FIRE 
was  given. 

NOTE  TO  SCORER:  Give  soldier  time  to  announce 
more  than  once.  Stop  the  time  when  soldier 
announces  SET,  but  give  him  time  to  repeat  SET 
before  you  announce  FIRE. 

4.  Sighted  on  target  within  IS  seconds. 


TIME  Initial  Retest 

Seconds  to  sight  on  direct  fire 

target  (from  range  element) :  First  Test  _  _ 

Repetition  2  _  ___ _____ 

Repetition  3  _  _ 


PM  NO-GO  Repetition  2 
PM  NO-GO  Repetition  3 
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DISASSEMBLE/ ASSEMBLE  BREECH  MECHANISM 


Equipment  Required  To  Set  Up  and  Conduct  Test 

M198  howitzer 
Flattipped  screwdriver 
1/2-inch  square  drive  socket  wrench 
TM9-1G25-211-10  with  change  2 


Procedures  To  Set  Up.  Station 

1.  Check  to  insure  breech  is  operational. 

2.  Lay  screwdriver,  socket  wrench,  and  TM  within  reach  of  the  soldier. 

Procedures  To  Be  Performed  Before  Testing  Each  Soldier 

1.  Return  breech  mechanism  to  operational  status  if  needed. 

2.  Lay  out  tools  for  soldier. 


Procedures  To  Conduct  Test 


1.  The  soldier  must  open  the  TM  to  the  section  on  breech  maintenance  before 
he  begins  to  disassemble  the  breech.  Give  him  about  one  minute  to  locate 
the  section.  If  he  does  not  locate  it,  show  him  how  to  find  it  in  the 
index.  He  does  not  have  to  look  at  the  TM  after  finding  the  section. 

2.  Start  the  time  for  disassembly  when  the  soldier  finds  the  section  in  the 
TM. 

3.  The  only  component  that  should  be  disassembled  in  this  test  is  the 
obturator.  If  the  soldier  starts  to  disassemble  any  other  component,  tell 
him,  "You  do  not  have  to  disassemble  that  component.  Continue  with 
disasembly."  (No  NO-GO.) 

4.  For  PM  3  (removing  the  firing  mechanism  housing  assembly),  the  soldier  may 

need  to  use  the  screwdriver  and  socket  wrench.  If  he  tries  and  is  unable 

to  do  those  steps  by  hand,  tell  him  to  use  the  tools.  Do  not  score  him 

NO-GO  if  he  is  able  to  do  the  steps  with  the  tools. 

3.  The  soldier  may  become  confused  with  the  format  of  the  TM.  If  he  cannot 
do  the  first  step  for  a  component  and  is  not  on  the  correct  page,  mark  the 
PM  NO-GO,  show  him  the  correct  page,  and  put  "TM”  under  COMMENTS.  If  he 
still  cannot  do  the  step,  tell  him  how  to  do  the  step  and  put  "prompt" 

under  COMMENTS.  If  he  still  cannot  do  the  step,  show  him  how  and  put 

"demo"  under  COMMENTS. 
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Scorer:  Soldier: 


Date: 


SCORESHEET 

DISASSEMBLE/ ASSEMBLE  BREECH  MECHANISM 


INSTRUCTIONS  TO  SOLDIER:  For  Chis  test  you  will  disassemble  and  assemble  the 
breech  mechanism.  First  disassemble  the  breech  and  obturator.  You  have  six 
minutes  for  disassembly.  Begin. 


PERFORMANCE  MEASURES  GO  NO-GO  COMMENTS 

1.  Opened  TM  to  section  for  breech  maintenance 

(p.  3-34  thru  3-46).  _ _  _ _ 

2.  Removed  the  M35  firing  mechanism.  _  _  _ 

3.  Removed  the  firing  mechanism  block.  _  _  _ _ 

4.  Removed  the  firing  mechanism  housing 

assembly.  _  _  _ 

5.  Removed  the  obturator  spindle  assembly.  _  _  _ 

6.  Disassembled  obturator  into:  disk,  rear 
split  ring,  inner  ring,  obturator  pad,  and 

front  split  ring.  _  _  _ 

7.  Completed  disassembly  in  six  minutes.  _  _  _ 


TIME 


Initial  Retest 


Minutes  to  disassemble:  First  Test 


PM  NO-GO  Repetition  2 
PM  NO-GO  Repetition  3 


Repetition  2 
Repetition  3 


SCORESHEET  (Cont’d.) 


DISASSEMBLE/ASSEMBLE  BREECH  MECHANISM 


PERFORMANCE  MEASURES  GO  NO-GO  COMMENTS 


INSTRUCTIONS  TO  SOLDIER:  Now  assemble  the  breech 
mechanism.  You  have  eight  minutes. 

8.  Assembled  obturator  so  that  split  rings  were 
180°  apart. 

9.  Placed  obturator  spindle  assembly  Into  breech¬ 
block  so  arrow  on  obturator  lined  up  with 
arrow  on  breechblock. 


NOTE  TO  SCORER:  Allow  about  two  minutes  for  the 

soldier  to  get  the  assembly  In  position.  If  not 

lined  up,  show  him  In  the  TM  how  assembly  should 

be  Installed.  Score  PM  9  NO-GO. 

10.  Tightened  assembly  by  turning  assembly 
counterclockwise  (In  direction  of  arrow 
on  breechblock. 

11.  Closed  breechblock  assembly. 

12.  Pushed  cartridge  extractor  towards  obturator 
spindle  assembly. 

13.  Rotated  breechblock  assembly  until  slot  was 
horizontal  and  extractor  was  against 
obturator  spindle  assembly. 

1A.  Installed  firing  mechanism  block  assembly. 

15.  Placed  firing  mechanism  in  block  assembly 
and  turned  It  1/A  turn  clockwise  while 
pressing  In. 

16.  Slid  firing  mechanism  block  assembly  to  the 
right. 

17.  Tested  action  of  firing  pin  by  pushing  and 
releasing  pin. 

18.  Completed  assembly  within  eight  minutes. 


TIME  Initial  Retest 

Minutes  to  assemble:  First  Test  ______  _ _ 

Repetition  2  _  _ 

Repetition  3  __ _  _ 

PM  NO-GO  Repetition  2  _ 

PM  NO-GO  Repetition  3  _ _ _ 
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APPENDIX  B 

DUTY  POSITION  AND  TASK  EXPERIENCE  QUESTIONNAIRE 
(FOR  TESTED  SOLDIERS) 


RECENCY  FREQUENCY  QUESTIONNAIRE 
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PERFORMED  TESTED  INDIV.  TRAINING  LECTURE  OR  DEMO  CREW  DRILL 

TASK  IN  OSUT  IN  OSUT  WITH  SUPERVISOR  (NOT  PERFORMED)  OR  RSOP  LIVE  FIRE 
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RECENCY  FREQUENCY  QUESTIONNAIRE  -  AUGUST  83 


NAME 

SSN 

GRADE 

BATTERY 

SECTION 

SECTION  CHIEF 

Date  Last  ARTEP 

Date  Last  Live  Fire 

_  Date  Last  Section  Drill 

Primary  Duty  Position.  (Circle  one  only) 

G  A/G  1  2  3  4  5  6  7  Dvr  Other _ _ 

Has  your  primary  duty  position  changed  since  you  were  tested  in  June?  Yes  No 

If  yes,  what  was  your  duty  position  in  June? 

Have  you  been  cross  trained  in  other  duty  positions  since  you  were  tested  in 

June  ?  Yes  No 

(Circle  the  positions  you  have  been  cross  trained  in) 

G  a/G  1  2  3  4  5  6  7  Dvr  Other _ 

Date  Xast  Cross  Trained  _ _ 

Additional  Duties  Performed  Last  2  Months.  (Circle  one  or  more) 

None  G  A/G  1  2  3  4  5  6  7  Dvr  Other _ _ 

Primary  Duty  Position  Last  ARTEP.  (Circle  one  only) 

G  A/G  1  2  3  4  5  6  7  Dvr  Other _ 

Duties  Performed  Last  Live  Fire  Exercise.  (Circle  one  or  more) 

G  A/G  1  2  3  4  5  6  7  Dvr  Other _ 

Duties  Performed  Last  Period  of  Section  Training.  (Circle  one  or  more) 

G  A/G  1  2  3  4  5  6  7  Dvr  Other _ 

Additional  Duties  Performed  Last  2  Months.  (Circle  one  or  more) 


a. 

Nuclear  Assembler 

f . 

Arty  Mechanic 

b. 

Security  Guard 

8- 

Heavy  Vehicle 

Driver 

c. 

RTO 

h. 

Light  Vehicle 

Driver 

d. 

Gun  Guide 

i. 

Other 

e. 

Ammo  Handler 

j. 

None 
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RECENCY  FREQUENCY  QUESTIONNAIRE  -  NOVEMBER  83 


NAME 

SSN 

GRADE 

BATTERY 

SECTION 

_  SECTION  CHIEF 

Date  Last  ARTEP 

Date  Last  Live  Fire 

_ Date  Last  Section  Drill 

Primary  Duty  Position.  (Circle  one  only) 

G  A/G  1  2  3  4  5  6  7  Dvr  Other _ 

Has  your  primary  duty  position  changed  since  you  were  tested  in  June?  Yes  No 

If  yes,  what  was  your  duty  position  in  Aigust? 

Have  you  been  cross  trained  in  other  duty  positions  since  you  were  tested  in 

August?  Yes  No 

(Circle  the  positions  you  have  been  cross  trained  in) 

G  a  /G  1  2  3  4  5  6  7  Dvr  Other _ _ 

Date  Xast  Cross  Trained 

Additional  Duties  Performed  Last  2  Months.  (Circle  one  or  more) 

None  G  A/G  1  2  34  5  6  7  Dvr  Other _ _ 

Primary  Duty  Position  Last  ARTEP.  (Circle  one  only) 

G  A/G  1  2  3  4  5  6  7  Dvr  Other  _ 

Duties  Performed  Last  Live  Fire  Exercise.  (Circle  one  or  more) 

G  A/G  1  2  3  4  5  6  7  Dvr  Other _ 

Duties  Performed  Last  Period  of  Section  Training.  (Circle  one  or  more) 

GA/G1  2  3  4  5  67  Dvr  Other _ _ 

Additional  Duties  Performed  Last  2  Months.  (Circle  one  or  more) 


a. 

Nuclear  Assembler 

f. 

Arty  Mechanic 

b. 

Security  Guard 

8- 

Heavy  Vehicle 

Driver 

c. 

RTO 

h. 

Light  Vehicle 

Driver 

d. 

Gun  Guide 

i. 

Other 

e. 

Ammo  Handler 

J. 

None 
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RECENCY  FREQUENCY  QUESTIONNAIRE  -  JANUARY  84 


NAME  _ _  SSN  _ _  GRADE 

BATTERY  _  SECTION  _  SECTION  CHIEF 


Date  Last  ARTEP  _ _  Date  Last  Live  Fire  _  Date  Last  Section  Drill  _ 

Date  Last  Tested  on  This  Project:  (Circle  one)  June  August  November 


Primary  Duty  Position.  (Circle  one  only) 

G  A/G  1  2  3  4  5  6  7  Dvr  Other__ _ 

Has  your  primary  duty  position  changed  since  you  were  last  tested?  Yes  No 
If  yes,  what  was  your  duty  position  when  last  tested? 

Have  you  been  cross  trained  in  other  duty  positions  since  you  were  last 
tested?  Yes  No 

(Circle  the  positions  you  have  been  cross  trained  in) 

G  A/G  1  2  3  4  5  6  7  Dvr  Other _ 

Date  Last  Cross  Trained _ 

Additional  Duties  Performed  Last  2  Months.  (Circle  one  or  more) 

None  G  A/G  1234567  Dvr  Other  _ 

Primary  Duty  Position  Last  ARTEP.  (Circle  one  only) 


G 

A/G  1  2  3  4  5 

6 

7  Dvr  Other 

Duties 

Performed  Last  Live  Fire 

Exercise.  (Circle  one  or  more) 

G 

A/G  1  2  3  4  5 

6 

7  Dvr  Other 

Duties 

Performed  Last  Period  of 

Section  Training.  (Circle  one  or  more) 

G 

A/G  1  2  3  4  5 

6 

7  Dvr  Other 

Additional  Duties  Performed  Last 

2  Months.  (Circle  one  or  more) 

a. 

Nuclear  Assembler 

f . 

Arty  Mechanic 

b. 

Security  Guard 

8* 

Heavy  Vehicle  Driver 

c. 

RTO 

h. 

Light  Vehf.cle  Driver 

d. 

Gun  Guide 

i. 

Other 

e. 

Ammo  Handler 

3- 

None 
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APPENDIX  C 


DUTY  POSITION  QUESTIONNAIRES 
(FOR  SUPERVISORS) 

JOB  BOOK  RECENCY 


APPENDIX  D 


SELF  REPORT  DATA 


SELF  REPORT  DATA 


At  each  test,  questions  were  asked  of  each  soldier 
regarding  the  last  time  any  of  the  tasks  was  performed 
(recency/frequency).  For  these  self  reports  we  asked  for 
the  most  recent  exposure  to  each  task  as  a  result  of 
individual  training,  lectures,  crew  drill,  ARTEP,  and  Live 
Fire  exercises.  Soldiers  were  also  asked  to  supply 
background  information  regarding  their  primary  duty 
position,  positions  in  which  they  were  crosstrained ,  and 
duties  performed  in  the  preceeding  two  months. 

Additionally,  between  each  of  the  retention  tests.  Section 
Chiefs  were  asked  to  complete  a  questionnaire  indicating 
last  ARTEP,  Live  Fire,  Section  Drill  and  duties  performed 
by  each  soldier  under  their  supervision.  Job  Book  data  for 
each  soldier  were  also  requested.  The  latter  provided  the 
date  of  the  last  individual  training  for  each  soldier  on 
each  task.  We  were  unable  to  retrieve  these  data  for  every 
soldier;  however,  we  were  successful  for  the  majority  of 
cases.  The  following  matrix  shows  the  number  of  soldiers 
for  whom  data  were  collected  from  these  sources  at  each 
retention  test. 


Retention  1  Retention  2  Retention  3 


Source 

Section  Chief  (or  Job 

Book ) 

Background 

50 

81 

0 

Recency/Frequency 

51 

61 

111 

Self  Report 

Background 

60 

89 

114 

Recency/Frequency 

60 

89 

114 

In  an  effort  to 

measure  the 

accuracy  of 

the  self 

reports  of  the  soldiers,  we  compared  these  to  the 
information  collected  from  the  Section  Chiefs.  We  measured 
agreement  between  the  two  in  the  following  manner:  For 
questions  regarding  duty  positions,  we  took  only  exact 
matches.  That  is,  if  a  Section  Chief  listed  a  soldier's 
primary  duty  position  as  Assistant  Gunner,  and  the  soldier 
reported  it  as  Gunner,  this  was  coded  as  a  no  match.  We 
allowed  more  flexibility  on  questions  involving  dates. 
Section  Chiefs  provided  exact  dates;  soldiers  usually 
responded  with  more  general  estimates,  such  as  "six  weeks 
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prior  to  testing."  We  considered  a  range  of  plus  or  minus 
two  weeks  as  acceptable  agreement.  We  found  a  high  level 
of  agreement  between  Section  Chiefs  and  self  reports  for 
primary  duty  position  and  crosstraining  positions;  at  each 
retention  test  there  was  at  least  80%  agreement.  There  was 
less  agreement  between  these  sources  regarding  duties 
performed  in  the  two  months  proceeding  the  test;  on  the 
average,  agreement  was  only  45%  on  these  types  of 
questions . 

We  were  particularly  interested  in  comparing  the  most 
recent  performance  of  specific  tasks.  Only  individual 
training  data  (Job  Book  vs.  self  report)  were  available  on 
a  task-by-task  basis.  Table  D.l  shows  the  percentage  of 
agreement  between  these  two  sources.  Retention  2  showed 
the  lowest  average  percent  agreement  (22.7%)  of  the  three 
test  periods.  However,  in  only  five  instances  did 
agreement  equal  or  exceed  50%.  At  Retention  1  and 
Retention  3  we  found  that  the  average  agreement  was  44.6% 
and  42.6%,  respectively.  When  there  were  disagreements 
regarding  whether  or  not  tasks  were  performed  since  the 
last  test,  data  from  the  Job  Book  were  slightly  more  likely 
to  indicate  task  performances  than  were  the  soldiers. 

After  reviewing  these  results,  we  decided  to  use 
Section  Chief  and  Job  Book  data  in  our  statistical 
analyses.  If  these  data  were  not  available,  self  reports 
were  edited  and  then  used.  The  editing  consisted  of  first 
examining  the  recency  dates  within  each  battery.  For  many 
of  the  tasks,  we  found  that  the  soldiers  in  a  given  battery 
practiced  together;  thus,  if  there  was  a  common  date  of 
task  performance,  that  value  was  assigned  to  all  soldiers 
in.  that  battery.  Also,  where  soldiers  reported  doing  a 
test  prior  to  their  last  testing,  that  value  was  scored  as 
"did  not  perform". 
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Table  D.1 

Percentage  Agreement  Between  Section  Chief  Data  and 

Soldier  Self  Reports 


Tasks 

Retention  1 

N  % 

Retention  2 

N  % 

Retention  3 
N  % 

Measure  Quadrant 

34 

52.9 

54 

25.9 

108 

45.3 

Set/Lay  Cannon 

35 

45.7 

54 

20.4 

107 

48.6 

Sight  Target 

34 

26.5 

54 

24.1 

106 

48.1 

Micrometer  Test 

26 

42.3 

32 

34.4 

41 

19.5 

Bores ight  Telescope 

33 

24.2 

54 

22.2 

101 

46.5 

Emplace/Recover 

Aiming  Posts 

34 

26.5 

57 

19.3 

109 

49.5 

Emplace/Recover 

Collimator 

34 

52.9 

57 

19.3 

108 

49.1 

Select  Ammunition 

34 

55.9 

57 

14.0 

108 

51.9 

Disassemble/Assemble 

M60  Machinegun 

34 

44.1 

58 

20.7 

107 

38.3 

Disassemble/Assemble 

Breech 

34 

50.0 

57 

26.3 

107 

43.9 

Install  Stakes/ 

Establish  Communication 

27 

48.1 

57 

22.8 

44 

29.6 

X 

44.6 

22.7 

42.6 
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